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ABSTRACT 

This report presents the results of a New Jersey 
school district field test of a comprehensive administrator training 
program. The program's main objectives were to develop a 
knowledge-utilization approach to the improvement of instruction and 
student achievement in basic skills, and to develop and test 
strategies for installing and disseminating the approach. Findings 
indicate that successful implementation varied somewhat according to 
the commitment and understanding of key concepts of both principals 
and teachers. Of the eight elementary schools and one middle school 
participating in the field test, two schools ranked relatively high, 
four medium, and three low in overall. levels of implementation. The 
moderate level of program implementation districtwide seemed to have 
positive results for students. Results indicated a reversal of 
previous trends, with students attaining around the national average 
in reading and exceeding the national average in mathematics, 
^achievement gains showed a direct correspondence to levels of program 
implementation. Extensive appendixes include results of related 
analyses in Pennsylvania and Delaware, copies of data collection 
forms/instruments, summary indexes of program implementation levels, 
New Jersey district/principal self-assessment results, and ^ 
implementation checklists for districts and principals. (JBM) 



*********************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. 

*********************************************************************** 



DOCUMENTATION REPORT: PHASE II 
OT" A FIELD TEST OF ACRIEV^ENT DIRECTED LEADERSHIP 

CD 



U.S. DEPARTMENT O? EDUCATION 

^ I NATIONAL INSTITUTE OF EDUCATION 

^ ^ EDUCATIONAL RESOURCES INF-ORMATlON 

' • CENTER <ERiC) 



This document has been reproduced as 



received from the person or organization 
originating it. 
Minor rhdngRS havp been made to improve 
reproduction quadly 

o Points of view or opinions stated in this docu . 
ment do not nei;e.^bt3nly represent official NiE 
posttioh or policy 



Thomas W. Biester 
Janice Kruse 

Erancine S. Beyer 
Ben A. Heller 



Basic Skills Component 
Research for Better Schools, Inc. 

444 North Third Street 
Philadelphia, Pennsylvania 19123 



g March 1983 



The work upon which this publication is based was funded by the National 
^nltltute of Education, Department of Education. The opinions expressed 
In tWs Jublication do not necessarily reflect the position or Policy of 
the Sationai Institute of Education, and no official endorsement by the 
National Institute of Education should be inferred. 



2 



ACKNOWLEDGMENTS 

This report presents the results of a field test of a comprehensive 
training program for administrators, principals, and. teachers called 
Achievement Directed Leadership. David Helms and Anna Graeber are the 
Director and Associate Director of the Basic Skills Component of Research 
for Better Schools, Inc., (RBS) which developed the program. The authors 
thank them for their guidance during the various stages of evaluation and 
writing of this report. 

The authors appreciate the contributions of the many cooperating 
educators in the field test . districts; without their assistance this study 
could never have been carried out. The authors also acknowledge their, 
debt to RBS field coordinators Janet Caldwell, Valarie French, and Barbara 
Smey-Richman. These coordinators played an invaluable role in the 
execution of the field test, and, in addition,- read drafts of this report 
for accuracy. 

Several other RBS staff members read drafts of this report and made 
suggestions for improvirg it. These persons include John Xonnolly , Deputy 
Director, Keith Kershner, Director of Research and Evaluation, and Louis 
Maguire, Director, Institutional Development. Louis Maguire also provided 
editorial assistance and supervision throughout the final phase 
of production. Past RBS employees deserving special acknowledgment for 
their work on the field test are Edgar Richards, director of field test 
activities during the main period of data collection, and Donald Coan, 
director of field test activities during the design,, planning, and initial 
data collection stages. Finally, the authors thank Francine Shelkinxwho 
typed interminable drafts and *'final revisions" with skill and goodwill\^ 

ill ' :•] . 



TABLE OF CONTENTS 



Page 



CHAPTER ONE: INTRODUCTION . . . • 

CHAPTER TWO: ACHIEVEMENT DIRECTED LEADERSHIP * 

The Classroom Variables , 

! The Improvement Cycle 

The Leadership Plan 

Staff Development 

CHAPTER THREE: FIELD TEST EVALUATION PROCEDURES • 

'Context for the Field Test 

Field Test Framework • ■ 

.Data Collection Procedures *■ 

Data Analysis Plans ' • • ' • 

Limitations of the Field Test • 

CHAPTER FOUR: FIELD TEST FINDINGS - NEW .JERSEY ..SCHOOL DISTRICT ... 

District Overview - ■ ••» 

The BSC Level ■ •■• • 

The District Level " • 

The Principal Level 

The Teacher Level * • • •• 

Summary of Program Installation and Implementation in the 

.New Jersey School District •• 

The Student Level 

Overall Summary of New Jersey Field Test Findings • 



1 
3 
3 
4 
7 

12 
17 
17 
18 
23 
26 
28 
35 
35 
39 
46 
52 
63 

70 

75 
94 



V 



Appendix A: 
Appendix B: 
Appendix C: 
Appendix D: 

Appendix E: 

\- 

Appendix F: 
Appendix G; 



Pennsylvania Analysis • 

Delaware Analysis • • • 

Data' Collection FonriG/Instruments •••••••••• 

Development of Summary Indices Describing 
Level of Program ^Implementation 

Self-Assessment Results , District/Principal 
Workshops - New Jersey School District . 

District Checklists 

Principal Checklist 



103 
117 
131 

205 

213 
219 
225 



CHAPTER ONE 
* INTRODUCTION 

The institutional mission of Research for Better Schools, Inc. (RBS) 
as an educational R&D' laboratory is to help elementary and secondary 
schools-in Delaware, New Jersey, and Pennsylvania improve their education- 
al programs (RBS, 1979). To accomplish this, RBS has cooperated with edu- 
cational agencies- at all levels in the ^tri-state area in developing and 
disseminating models for school improvement within the context of state- 
wide school improvement efforts. . • 

The Basic Skills Component (BSC) of RBS has adopted the mission of 
^ helping schools in Delav^are, New Jersey, and Pennsylvania to improve their 
basic skills programs. -^ Pursuant to 'this mission and with the support of 
the .National Institute of Education (NIE) , the BSg launched a basic skills 
improvement project in'l977. The^project has tW6 main. objectives : (1) to 
develop a knowledge-utilization approach to the improvement of in^fexjiction 
J.and, ultimately, student achievement in the basic skills; and (2) to 

develop and' test strategies for installing and dissemins^ting the approach. 
Over the course of the project, many educational -agencies have cooperated 
with the Basic Skills' Component . The advice and assistance contributed by 
these partners have been critically important to the accomplishment of the 

project's objectives. - v 

The result of the project's cooperative development work is Achieve- 
ment Directed LeadersUp, a staff development program aimed at helping^^ 
V teachers, principals, and district administrators use knowledge and re- 

search findings to improve basic skills instruction, and, ultimately, stu- 
dent achievement. This document reports on the field test of 
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Achievement Directed Leadership undertaken by the BSC in response to 
recommendations-of an NTE^ site review team and the NIE project officer, 
Dr. Barbara Lieb-Brilhart. ' The overall purpose of the field test was to 
assess the effectiveness of Achievement Directed Leadership. The field 
test hypothesis was that Achievement Directed Leadership has a direct and 
significant effect on instructional leadership and classroom processes/ 
conditions, and consequently, on students' basic skills achievement. To 
test .his hypothesis, the program was fully implemented in a school dis- 
trict in the tri-state area served by RBS, and a large body of data was 
collected both on program implementation and outcomes in this district. 
The second chapter gives a brief overview of Aphievement Directed 
' Leadership. The third' chapt^er details the evaluation "methods and limita- 
tions of the fie^d test. "The fourth chapter discusses implementation and 
outcomes of Achievement Directed Leadership in the field test district. 
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CHAPTER TWO ^ \ . 

■ " ' \ 

ACHIEVEMENT DIRECTED LEADERSHIP \^ 

\ 

Achievement Directe 1 Leadership is a program aimed at helping 
teachers, principals, and, central office. staff use research knbwledge to 
improve basic skills instruction, and, "ultimately, student achievement in 
elementary schoo],s. It has four mainj'.elements : (1) a f-'.us on .a set of 
classroom variables that seems to be especially important to basic skills 
achievement;- (2) a variables majiagement strategy, or "improvement cycle"; 
(3) a method, of coordinating and focusing improvement efforts across the 
levels of the district hierarchy, called the "leadership plan":; and (4^ a 
staff development program which provides the training necessary for 
installation and maintenance of the leadership plan. The next four 
•sections of thir. chapter 'describe each of these elements of Achievement 
Directed Leadership. . ^ 

The Classroom Variables . 

Recent research findings pinpoint Kany variables in elementary school 
classrooms which are significantly related to student achievement (e.g., 
Brophy & Good, 1974; Dunkin & Biddle, 1974; Medley, 19?'?; Rosenshine & 
Furst, 1973). These findings indicate that students who have^ or acquire 
,,rior learni ng that helps them to learn new content and who spend an 
adequate amount of engaged time successfully coveri ng, mastering, and 
reviewing content on which they will be tested are likely to perform 
better on year-end achievement tests' than students who do not act this 

V 

way. , \ 



All educators in a diatrict, e'specially teachers, aro-urged to give 

special attention. to the following student behaviors or focus variables:. 

• Prior learning , knowledge possessed by students which will 

facilitate their learning of new subject matter (Bloom, 1976; . 
Carroll, 1963) •■ , 

' • " Student engaged time , amount of time students actually speAd on 
assigned learning tasks (Anderson, 1981; Carroll, 1963; Fisher,- 
Marliave, &'Filby, 1979; Rosenshine, 1979; §tallings & Kaskowitz, 
1974) ' 



• Academic performance , success students experience witn daily 
learnihg^ tasks, their master^^ -of .curriculum units, and their ^ 
review of content achievements (Block & Burns , 1976;' Bloom, 1976; 
Crawford, 1978; Fisher, Marliave, & Filby, 1979) 

« ^Coverage of criterion-relevant content , opportunity students have 
to learn the content on which they will be tested (Cooley & 
Leinhardt, 1980; English, 1980) • 

Exclusive attention by -educators to We or another of the -focus vari- 
ables . without due attention to all will not be -fully beneficial. Further.- 
more, these four variables az&, in turn, influenced by myriad other 
variables. How the four focus and other variables are addressed/by 
Achievement Directed Leadership is described in the discussion of the 
im'provement cycle that follows. 

t,. " The Improvement Cycle 

No matter how valid the r'^search, knowledge of significant classroom 
variables is often not sufficient for the improvement of instructional 
practices. In qr4er ^o effect change, a practical method of applying this 
research to particular classroom settings must be included in the 
knowledge resource. The variables management process, or improvement 
cycle, is such a method, and in Achievement Directed Leadership it is 
applied to each of the focus variables. 




The improvement cyctH-is a f our-phase iteratjlve pr<)<3ess by which 
educators can identify and take advantage of op^rtunli^s to improve , 
classroom instruction, variable by variable. Phase?, on'eai>d two o.f.the. 
•'improvement cy'cle are concerned |Witlv student classroom behaviors , while^ 
/phases three and four deal primarily W^th teacher behaviors,. The cycle is" 
'depicted graphically in- Figure 1 • •, ' , ' 




" Ph«e III 



Figure Improvement cycle used to manage e 



lassrooih variables related to 



students' achievement in reading/language arts or mathematics. 




The following description illustrates how the improvement cycle is 
applied \o one of the -focus variables, student engaged time. A teacher, 
working with the principal or another educator, attempty^T^s^^S-that 
student engaged time is at a level conducive to high student achievement, 
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In phase ope the principal or assisting teacher collects data on engaged 
time in the 'teacher 's classroom in a manner similar to the way data were 
collected in relevant research studies. By comparing these data with data 
from classrooms resBa^ch studies, the teacher and principal, or other 
assisting educator, determine in phase two whether an improvement oppor- 
tunity exists. , 

■ ■ If they find that the level of 'student engaged tike is already *t an 
•ideal level, they would, not" complete the .remaining phases of the cycle, 
but instead. would go ^ack to phase one, scheduling dates for subsequent 
data .collection. ^If improvement is' possible, they decide in phase three 
upon a strategy to effect the neces'sary change. In the third phase the - 
involved instructional leaders take i'nto account many other classroom 
variables that may affect student achievement indirectly through their 
impact on the targeted focus variable. , After preparing to implement the 
lassroom modification, tfhe teacher proceeds to implement and monitor the^ 
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change in phase four. 

Instru'ctional, leaders are able to assess the effectiveness of the 
classroom modification by repeating phases one and two, wherein they 
collect and analyze new .data after dn appropriate interval of^ time has 
elapsed. For example, if the strategy has h4d little or, no effect on. 
Students' engage'd time, they would proceed to phases three and four again, 
adjusting "the classroom modification or introducing a new modification as 
cireumstances dictate.' . ^ 



The Leadership Plan 

The leadership pUn is a means by .which central office and school 
staff can coordinate and focus their efforts to establish, and maintain 
instructionally effective- classrooms . This plan, derived from selected 
studies on effective classrooms, schools, and districts, specifies several 
role-related functions for educators at each level of the school district, 
and makes explicit channels of communication between levels. Each level 
performs its functions using the improvement cycle and focuses on the goal 
of improving, or maintaining, the classroom focus variables. The follow- 
ing three sections discuss how the leadership.plan operates at the 

^ ■ . 

classroom, school, ^nd district levels. 

^> . ■ 

Classroom ^ 

' The leadership plan emphasizes the importance of the follox^ing 
teacher functions: planning classroom activities and procedures, managing 
the classroom, and delivering instruction. Since research indicai,_es that 
students' achievement is vitally telated to their classroom behaviors 
(i.e., the focus variables), the leadership plan calls for teachers to 
give these behaviors special' attention by performing their role-related 
functions according to information supplied through the improvement cycle. 

Figure 2 ii a graphic representation of the leadership plan for the 
classroom level. As shown, students' classroom behaviors are signifi- 
cantly influenced by their entering behaviors, especially those which 
.reflect their prior academic learning. According to the leadership plan, 
teachers take these entering behaviors into consideration as they plan 
..instruction for students. This is represented in Figure 2 by the solid 



arrow connecting students' entering behaviors with the teacher.. Further- 
ntore. the teacher uses the improvement cycle to attend to all other class- 
room variables as they plan, manage the classroom, and give instruction. 
The arrow connecting teacher and classroom is double headed, however, to 
indicate that each influences the other .Finally . students' year-end 
achievement is directly related to their classroom behaviors. Figure 2 
recognizes this, while also taking into account the relationship between 
• students' entering behaviors and their year-end achievement. 
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Figure 2. The leadership plan: the classroom level. 

. Teaching in the classroom is a complex process., and it occurs in the 
context of larger and even more complex settings, the school and district, 
which frequently influence the conditions and processes of the classroom. 
The leadership plan calls for teachers to regularl^operate with the 
principal and other teachers in planning for and implementing improvements 
t the school and district. levels as well as in;their, respective class- 



a 

rooms. 
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School ' ^ 

Although research has not yet made clear the relationship between 
principals' leadership and classroom instruction (Koehler, 1981), some 
research, and the experience of the BSC and its projegt partners, suggests^ 
that several kinds of principal support are needed to maintain at the 
classroom level the type of instructional leadership described in the 
previous section. This support derives from principals' performance of 
the following 'functions: planning for and with teachers, training 
teachers, and providing participatory supervision to teachers. 

These principal functions are intended to facilitate teachers' use of - 
the improvement cycle and foster teacher growth. Figure 3 represents the " 
use of the 'leadership plan for the school, and shows the relationship of 
the principal to the teacher and classroom. The arrow froiB classroom to 
principal indicates that the principal is continually informed of class- 
room conditions and processes through regular classroom visits, teacher 
reports, and participatory supervision activities with the teacher in the 
use of the improvement cycle. The double-headed arrow connecting 
principal and teacher represents a two-way flow of information. This 
exchange of information occurs in regular principal/teacher conferences. 
The leadership plan calls for these conferences to be held frequently, and 
, to include review of the classroom data on each focus variable. 

District 

Something is known of the characteristics of effective districts and 
of the critical elements that contribute to a district's success in 
implementing planned change (e.g., Berman & McLaughlin, 1975;.Pincus & 



Williams, 1979)- However, research and documented knowledge have little 
specific to say about how these factors affect instructional leadership in 
schools and classrooms. It is the experience of the BSC and its partners 
that several kinds of central office support can help to establish and 
sustain the kind of instructional leadership described above at the school 
and classroom levels. 
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Figure 3, The leadership plan:- the school level. 

The functions of central office staff are similar to those of princi- 
pals, and are equally concerned with the Classroom dimensions which affect 
student achievement. These functions , however , are primarily directed to 
the support of principals. The central office functions are: planning 
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with principals, training principals to perform their role-related 
functions, and providing participatory supervision to principals- These 
central office functions are intended to facilitate efforts of principals 
to promote and support growth of teachers as instructional leaders- 

Figure 4 depicts the relationship of district leadership to the prin- 
cipal, teacher, and classroom- The solid double-headed arrow between 
principal and district indicates a two-way flow of imormation- Although 
much of the communication will be informal, the principal and district 
leadership should also have formal conferences in which they review the 
documented outcomes of the principal's conferences with teachers. 
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Figure 4. The leadership plan: the district level, 
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District leadership and the principal give explicit attention during their 
own conferences to the status of classrooms with respect to the focus 
variables and to the teachers' plans for and success in improving instruc- 
tion. During the conferences and throughout the year, the .instructional 
leadership plan calls for the district leadership to be continually „alert 
to opportunities to assist principals with the conduct of their own • 
leadership responsibilities and functions. 

The arrow from students' year-end achievement to the district indi- 
cates the importance of these assessment data to central office staff. 
For example, during district/principal conferences, district leadership 
evaluates the classroom information compiled by the principal in terms of 
students' past achievement and district goals for their achievement in the 
current year. 

Although the district relies primarily on the principals for informa- 
tion concerning schools and classrooms, district leadership may also 
acquire information directly through personal visits and reports. The 
arrows in Figure 4 from classroom and teacher to district leadership 
acknowledge this, and are broken to indicate that central office staff's 
visits to classrooms and with teachers are necessarily much less frequent 
than those of principals. 

Staff Development 

In order to implement Achievement Directed Leadership, educators need 
to (1) be informed of the program; (2) understand and develop skills in 
the use of instruments and procedures associated with the impr^ovement 
cycle; and (3) plan for implementation and maintenance of the program in 
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their own organisations. In short, they require a program of specialized 
staff development. The fourth main element of Achievement Directed 
Leadership is such a program. 

The staff development program is not canned, and canndt be effec- 
tively implemented by instructional leaders who use the. program materials 
simply as recipes.- Initial training and support should be acquired from 
an external agency. However, school districts should not depend continu- 
ally upon outside assistance. The major thrust of the staff development^, 
program is to assist school districts and sdjiools to acquire their own 
capability to sustain Achievement Directed Leadership in their organiza- 
tions, and, if they choose, to lend initial implementation assistance to 
other school districts and schools. \^ 

The orientation to Achievement Directed Leadership is short and 
should be given by a knowledgeable session leader. The purpose of this 
orientation experience is to inform prospective users about Achievement 
Directed Leadership and the staff development program. The decision to 
adopt all, or part, of the program may require additional information and 
discussions. 

There are two kinds of training which help educators develop under- 
standing and skills in the use 6f Achievement Directed Leadership. One 
kind trains instructional leaders in the management of classroom variables 
and is divided into two separate packages, one relating to content vari- 
ables and the other to time variables. The other provides instructional 
leaders with . training specifically related to their roles and functions in 
the leadership plan. 
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Training in the management of the content variables focuses on the 
importance of prior learning and coverage of criterion-relevant content 
and describes the activities of the improvement cycle associated with each 
variable. Workshop experiences prepare district staff and principals to 
assist teachers in the preparation and use of integrated- instructional \^ 
plans that attend to these two variables. These plans also provide a 
means for instructional leaders, including teachers, to monitor content 
coverage and mastery of content during the course of the year. 

Training also prepares all instructional leaders to be competent 
managers of student engaged time. Portions of the training are presented 
on videotape. Precoded videotapes of actual classrooms provide practice 
in classroom observation. On completion of training,' trainees can collect 
data on student engaged time, compare their data with data from similar 
classrooms reported in research studies, select research-based classroom 
strategies to exploit identified opportunities for improving, studeiit 
engaged. time, and' implement and monitor the classroom modifications. 

Although current training does not treat students' academic perform- 
ance as a focus variable, the approach recognizes the critical importance 
of this variable. In' the third phase of the instructional improvement 
cycle, special attention is given to the success of students with daily 
work, to their mastery of curriculum units, and to their maintenance of 
previous achievements. • ^ 

It is recommended that Achievement Directed Leadership be implemented 
district-wide, and to this end, separate versions of instructional leadef- 
ship training are provided for educators at each level of the district.. 
Training makes clear the roles of the respective levels in. the leadership 
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plan. It also details the functions of each level, provides background 
information relative to each function, and suggests ways leaders can 
facilitate performance of these functions. In short, this training is 
designed to aid all instructional leaders to coordinate and focus their 
efforts to establish, support, and sustain effective classroom instruction 
withifi effective schools and districts. 

Training in the leadership plan is not a brief intensive experience 
as is the Initial training for management of the classroom variables. 
Rather, leadership plan training is an on-the-job, continuing experience 
in the solving of real ..problems associated with the implementation and 
institutionalization of Achievement Directed Leadership. 

Since problems associated with implementation and institutionaliza- 
tion are often shared across levels of the school district, the leadership 
training is designed to assist the school district with the planning and 
conduct of leadership seminars or conferences that address these problems 
on a cross-level basis. Although the assistance of an external consuit-ant 
may be required to plan and conduct early leadership seminars, these 
experiences are intended to prepare leaders to assume early management of 
their own planning and problem-solving sessions. 
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CHAPTER THREE 

FIELD TEST EVALUATION PROCEDURES 

This chapter describes the procedures used in conducting the field 
test of the basic skills instructional improvement approach during the 
1981-1982 school year. Separate sections describe the general context for 
the test, present the conceptual framework ^f or the study, indicate the 
data collection procedures that were employed, note data analysis plans, 
and discuss study limitations. 

Context for the Field Test 

During the late spring and summer of 1981, districts which might 
serve as sites for the field test were investigated. One main site was 
selected as the focus of the field test.*^ This site, referred to as the 
New Jersey School District, agreed to mount a reasonably full implementa- 
tion of Achievement Directed Leadership during the 1981-82 school year. 

The New Jersey School Distract had cooperated with RBS in the 
development of the BSC approach for three years prior to the field test 
year. Developmental efforts were undertaken with a few volunteer teachers 



^During the 1981-82 school year, the BSC also worked with and 
collected information from two other school districts (one in Pennsylvania 
and one in Delaware) in regard to implementation of Achievement Directed 
Leadership. Each of these two districts represented an opportunity to 
test supplementary questions about the effectiveness Sf Achievement 
Directed Leadership^ ^A. report on each district's ut^ilization of the 
prdgram is found in the appendices (Appendix A for the Pennsylvania 
District and Appendix B for the Delaware district). 
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in each 'of four elementary schools. The field test represented expansion 
and full implementation of the Achievement Directed Leadership program in 
the district's eight elementary schools and one middle school. Implementa- 
tion-plans indicated that the district would endeavor to reasonably 
replicate all elements of the Achievement Directed Leadership program. 

Field Test Framework 

Table 1 presents the overall frane^yprk that was used to guide the 
field test. The , framework contains five levels: (1) the BSC; (2) central 
office staff/district (e.g., superintendents, assistant superintendents, 
curriculum supervisors/coordinators); (3) principal/school; (4) teacher/ 
classroom; and (5) student/classroom. At each level, the functions listed, 
in the' table refer to the prescribed activities for that level that«are 
deemed essential for implementing- the approach. At the BSC level the 
primary functions are planning and conducting initial orientation of dis- 
trict staff to the improvement approach, planning and delivering training 
to district staff in the/use of the appr oarch , and providing follow-up 
technical assistance to ensure and upgrade the quality of implementation. 
Essentially, these three functions are concerned with the process of 
initially installing and maintaining the improvement approach in the field 
test sites. Ideally, implementation of Achievement Directed Leadership 
helps school districts to develop the capacity to establish and sustain 
the instructional improvement approach. Appropriate central office staff 
should acquire a strong commitment to the program, as well -as the knowl- 
edge and skills needed to implement it. Therefore, an important objective 
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. Table 1 
Field Test Fra^mework 




District 



Field Test 
Obiectives 




Field Test 
Methods 



• Plan and conduct district 
orientation to the improve- 
ment approach 

• Plan and deliver initial 
training to district staff 
in the use of the approach 

• Provide follow-up technical 
assistance to di'strict to 
facilitate iniplementation of 
the approach 



• Plan and conduct principals' 
orientation to the improve- 
ment approach 

• Plan and conduct training 
of principals in use of the 
approach (i-e., their role- 
related functions) 

• Engage in participatory- 
supervision with principals 



• Document the process of 
^ initially installing the 

improvement approach 

• Document 'the outcomes of 
installing the improve- 
ment approach in terms 
of acquired knowledge, 
skills and attitudes 



• Observe orientation and training 
sessions 

• Observe Implementation Seminars . 

• Interview participants (i-e,, dis- 
" trict leadership and supervisots) 

• Survey participants' 'reactions 
to orientation/training 



•Observe orientation and training 
sessions 

• Interview district staff and prin- 
cipals about district/principal 
supervisory conferences 

• Analyze District/Principal Con- 
^^--ference Form 

• Survey principals' reactions to 
orientation/ training ^ 



Document the process and 
outcomes of staff develop- 
ment for principals 

Document the process and 
outcomes of participatory- 
supervision 
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Principal • Plan and conduct teachers' 
orientation to the improve- 
mf at approach 

• Plan and conduct training of 
teachers in the use of the 
improvement cycle/focus 
variables 

» ^Engage in participatory- 
supervision with teachers 



o 



Level 



Table 1. (continued) 
Field Test Framework' 



Functions 



Field Test 
Objective 



Field Test 
Methods 



• Document the process and 
outcomes of staff develop- 
ment for teachers 

• Document the process and 
outcomes of participatory 
supervision 



o Document principals^ use 
of the imp'rovement cycle 



• Observe orientation and trainitig . 
sessions 

• Interview principals and teachers 
about district/principal supervi- 
sory conferences^ 

• Analyze the Principal/Teacher 
Conference Form 

Survey teachers' reactions to 
orientation/ training 

• Validate classroom observations. 



,Teacher • Plan lessons and classroom 

activities 

t Manage 'the classroom 

• Deliver instruction 



• Document teachers' use of 
the improvement cycle/focus 
vatrf-ables to guide planning, 
management, and delivery of 
instruction 



• Interview teachers about use of 
the improvement cycle and its im- 
pact on plannl^ig, management , ^ and 
delivery of instruction 

• Obtain confirmatory reports from 
principals, district. staff , and' 
BSC field staff of teachers' use 
of cycle , ' 

• Survey teachers' use of improve- 
ment cycle / 








Table'^^^continued) 
Field Test Framework 




Level 



Functions 



Field Test 
Objective 



Field Test 
Methgds 



Student • Demonstrate student class- 

room behaviors identified as 
critical to achievement 

# Demonstrate appropriate 
levels of achievement in 
basic skills subjects 



• Document impacts of the im- 
provement approach on criti- 
cal student behaviors 

• Document impacts of the 
improvement approach on 
students' achievement in 
reading/language arts and 
mathematics 



• Analyze student-behaviors: student 
engaged time; itrstructional over- 
lap; academic performance; and 
prior learning 

•Analyze 'students' achievement 
based on standardized achievement 
tests used in the district 
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of the field test was to document both the processes and outcomes of the 
Bsc's work with the district level. 

At the second and third levels within the field test framework, 
district leadership and principals share common functions; planning, 
training, and supervision. Ideally, central office st^ff.are responsible 
for planning and conducting the principals' orientation and training and ^ 
for providing participatory supervision to principals with respect to the 
principals' responsibilities. Principals, in turn, are responsible for 
planning, training, and supervising teachers. The field test objectives 
at the district level were to d^ocument the processes and outcomes of 
principals' staff development and central office participatory supervision 
of principals. * At the principal level, the objectives were to document 
the processes and outcomes of staff development for teachers, participa- 
tory supervision of teachers by principals, and principals' use of the 
improvement cycle. 

The primary functions at the teacher level are planning, managing the 
classroom, and delivering instruction. Utilized properly, the improvement 
cycle ca^n aid the' teacher to improve his/her performance in these func- 
tional areas. The field test objective at this level was to document 
teachers' use of the improvement cycle and focus variables in performing 
these functions. 

.At the fifth level of the framework, students are expected to demon- 
strate behaviors that promote effective learning (i.e., behaviors associ- 
ated with the BSC focus variables) and to perform at appropriate levels of 
achievement in basic skills. At' the student level the field test was pri- 
marily concerned with the assessment of expected outcomes of the approach, 
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namely, improved student classroom behaviors and improved achievement in 
basic skills subjects. 



Data Collection Procedures 



, As indicated in the last column of Table 1, multiple data sources 
were used to address the field test objectives at each of the five levels. 
The variety of sources is indicated in Table 2. Due to resource con- ^ 
straints and feasibility considerations, most of the information was 

0 

self-report in nature. In order to enhance the reliability of these data, 
attempts were made to "triangulate" wherever possible. Several types of 
'educators (e.g., superintendent, principal, teacher) were questioned about 
the same topics in order to gain insight into their various perspectives. 
Consistency of findings from different sources adds credibility to argu-- 
ments of effectiveness. Each of the data sources is briefly described 
below. A copy of many of the forms/instruments is included~~in Appendix C. 

• ' District/School Documentation ; Proposals, plans, correspondence, 

reports, and memos were collected fro,m the district and reviewed. 

• BSC Contact Reports ; This was a formal process ...for documenting 
all BSC field work. The reports describe objectives, activities, 
outcomes, and future plans for each contact. 

• BSC Observations and Informal Interviews ; BSC observers attended 
some orientation and training sessions and made impressionistic 
tlotes.' Other informal observations and discussions occurred 
throughout the course of the field test. 

0 Post-Training Questionnaires ; Specific questionnaires tailored to 
each training event solicited participants' reactions to training, 
self-assessments of acquired knowledge and skills, and general 
comments. ; 

« Post-Training Interviews ; Interviews were conducted with district 
staff and principals to obtain in-depth information regarding the 
effectiveness of training and their perceptions of the Achievement 
Directed Leadership approach. . . 
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Table 2 



Summary of Field Test Data Sources, by Level 



Data Sources 



District/School Documentation 
BSC Contact Reports 

BSC Observations and Informal 
Interviews 

Post-Training Questionnaires 
Post-Training Interviews 
District/Principal Conference 
Forms 

Preliminary and Final Superinten- 
dent Interviews 

Final District Staff Interviews 

Final Principal Interview 

Principal Questionnaire 

Principal Participatory Supervision 
Interview 

Principal/Teacher Conference 7orm 

Teacher Questionnaire 
Final Teacher Interview 
Instructional Overlap Forms 
Classroom. Observation Forms 
I standardized At;iaeyement Tests 



Level of Analysis /j^'- 



BSC 
Level 



0 
X 
X 

X 
X 



District 
Eevel ' 



NOTE: X indicates "primary data source 

0 indicates secondary data source 



X 



X 



X 
0 



Q 
0 



Principal 
Level ' 



X 
X 

•X 



X 



0 
X 
X 



0. 
0 



Teacher 
Level 



0 
0 



0 



0 



X 



0 
0 



Student^ 
Level 



0 
0 



0 



0 



■X 
X 
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, Di strict/Principal Conference Forms ; This form was used by 
district leadership and principals to identify opportunities to 
develop effective strategies for instructional improvement. 

, Field Interviews with Superintendent. District Staff , Principals, 
and Teachers: Specific interviews tailored to each grtoup were 
administered at the. end of the school year. These were generally 
designed to elicit information about roles and activities during 
the year, perceived success and problems, and overall reactions to 
the approach. The superintendent was also intervteWed at the 
beginning of the field test. 

, - Principal and Teacher Questionnaires ; I^orms (primarily close- 
ended) directed to each specific group were designed to obtain 
information on the extent and quality of implementation and 
reactions to the approach. These were administered once towards 

• the end of the school year. 

, P rincipal Participatory Supervision Interview ; Special interviews 
were designed to describe principals' experiences in using the 
improvement cycle and in conducting participatory supervision with 
teachers. 

, P rincipal/Teacher Conference Form : This form was designed to help 
principals and teachers implement the improvement cycle and con- 
duct participatory supervision conferences. The form summarizes 
information on prior learning, instructional overlap, success and 

' mastery levels, engaged and allocated time, and engagement rate. 
' This information was to be used to identify opportunities for 
improvement. 

• 'instructi onal Overlap Forms : Various forms wpre used to summarize 

the degree to which each teacher's instruction matched the content 
on the appropriate achievement test. 

, Cl assroom Observation Forms ; These, forms were used to structure 
observations and to describe and calculate allocated time, engaged 
time, and engagement rate. 

• Standardized Achievement Tests: These tests are part of the 
district's testing program and are administered in spring each 
year to all students. The New Jersey School District used the 
California Achievement Test (CAT). In addition, scores from the 
statewide basic skills competency test were available. Total 
reading and mathematics .subscores were collected ; students took 



"^Scores for special education and English as a Second Language 
(ESL) students were not included in the collected data. 




the level of the test designated by the test publisher as appro- 
priate for their grade. 

It is clear from this wide variety of information sources that the scope 

of the overall field test data base is quite large. 

^ Data Analysis Plans 

The guiding principle in the data analysis was reduction of the 
enormous quantity of information to a simple, straightforward, and 
meaningful level. Since much of the data was qualitative and/or self- 
report, the analyses were primarily descriptive. Interviews were 
abstracted using a structured format,., and general themes were reported in 
narrative fashion. Survey checklists and observation data were summarized 
in terms of descriptive statistics, including frequencies, means, and 
percentages. Quantitative data were collapsed into meaningful categories 
wherever possible. • 

General indices were constructed to summarize implementation and 
classroom process data. 'The implementation index synthesized information 
on principal and teacher attitudes and behaviors related to use of the key 
elements of the Achievement Directed Leadership program. The classroom 
process indices related student engaged time and instructional overlap 
data to existing research findings in terms of expected levels of student 
achievement. Both implementation and classroom process indices were 
developed to examine the relationship between these variables -and student 
achievement. The procedures for deriving implementation indices are 
found in Appendix D. Because the validity and reliability of these 




indices are subject to question, caution must be exercised in interpreting 
results. 

The primary level for. reporting implementation and achievement data 
was the school. Where appropriate, grade level data within a school were 
examined. Analysis of classroom data would have been ideal, and attempts 
were made to collect data at the student level. However, initial analyses 
indicated severe data completeness problems and lack of representative- 
ness. Analyses at the school level can be made with reasonable confidence 
since there were numerous data points and sufficient sample sizes. 

Analysis of achievement data was guided by the Title I norm- 
referenced evaluation model (model AlJ see Tallmadge & Wood, 1976). This 

• * *■ 

model compares students' standing relative to national norms at one point 
in time to their standing relative to national norms at a subsequent point 
in time. All things considered, position relative to the norms should not 
substantially change across test dates. Introduction of an appropriate 
educationai intervention is hypothesized to account for changes in rela- 
tive student achievement observed. Normal curve equivalents (NCEs) were 
used as the basis for analysis. These are standardized scores with a mean 
of 50 and a standard deviation of 21.06. Scores were collected for each 
participating field test school at each grade within a school for 1980, 
1981, and 1982. ~ A "diagonal'' analysis of results was conducted, i.e., 
results of students in grade three in 1982 were compared with results of 
students in grade two in 1981 .and grade one in 1980. The comparison from 
year to yedr is, therefore, essentially for the same groups of students. 
Scores do not exist, of course, for field test second graders before 1981, 
or for first graders before 1982. Therefpre, school means spanning the 
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three years (1980, 1981, 1982) do not include scores for these two grades. 
Change from 1980 to 1981 was used as a baseline to compare with gains 
registered from 1981 to 1982. , ■ , 

There are no hard and fast rules for interpreting the significance of 



NCE gain scores. Some evaluators suggest a change of one-third of a 
standard deviation (i.e., seven NCE points),.as a rule of thumb for 
educational significance, while others note that any change (i.e., one NCE 
point) is good (Tallkadge, & Wood,' 1980; Tallmadge, 1976). Examination of 
the -standard error of the difference between means for samples in this 
study suggested that a. change of five NCE points from year to year could 
be regarded as an appropriate rule of thumb for determining the educa- 
tional significance of gains. 

' Limitations of the Field Test 

Due to the developmental nature of the Achievement Directed Leader- 
ship program, as well as feasibility and resource constraints, no appro- 
priate control or comparison groups were available for the field -test. 
Thus, the evaluation was not rigorous in an experimental sense. The 
design can be categorized as nonexi^erimental rather than true or quasi- 
experimental (Campbell & Stanley, 1966). A nonexperimental evaluation 
design does not imply, however, that the test was performed in a haphazard 
way or that it "did not provide indicators of the success or failure of the 
BSC approach. On the contrary, a wealth of consistent information was 
gathered that, collectively, lends credence to beliefs about hypothesized 
program effects. Although indicators of descriptive or correlational 
relationships are not as credible as • indicators of experimental, causal 



3t^ 28 
0 



relationships, they, nevertheless, provide support for arguments 
concerning program effectiveness. 

Limitations to the field test evaluation fall into four major 
categories: (1) timing issues, {2) design issues, (3) analytic* issues, 
and (4) data issues* Any interpretation of field test results must - 
recognize the implications of the issues. Each of the issues is discussed 
below in more detail. 

Time issues ■ ' 

The Achievement Directed Leadership program is a large scale and 
fairly complex educational innovation. Such an innovation requires a con- 
siderable length of time to become fully implemented in a school district. 
It is not a program that can be "plugged in" at the start of the school 
year; rather, a gradual and extended implementation process should be 
employed. For the field test, it ^ was expected that full implementation 
would require at least an entire school year. According to the developers 
of the approach, significant changes in student behavior and achievement 
might not occur in the first year of im.pletnentation of the approach since 
practitioners would still be acquiring proficiency in their roles and 
functions. A more realistic objective for a one year implementation is 
the helping of educational practitioners, particularly 

principals and teachers, to establish the proper conditions in schools and 
classrooms which research indicates will ultimately facilitate improvement 
in student learning. Nevertljeless , the BSC considered the assessment of 
student outcomes to be a foremost objective of the field test. 
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Design Issues 

A second major limitation relates to the overall evaluation design. 
Perhaps the most critical problem in the field test is whether any ' 
observed changes or improvements in student behavior and achievement can 
be attributed solely to the effects of Achievement Directed Leadership or 
to other factors occurring in the field test district. There is wide- 
spread agreement among evaluation researchers that the most powerful way 
of demonstrating that an intervention has a real effect on some outcome is 
to employ an experimental design in which, typically, pre and posttests 
are given to a 'randomly selected group receiving the treatment and to other 
randomly selected groups receiving either no treatment or some variation. 
However, an experimental or quasi-experimental design was not appropriate 
or feasible for the field testfor the following reasons. .First, the .New 
Jersey School District was interested in adopting the improvement approach 
on a district-wide basis. • Under this condition, there would be no classes 
or schools within the district that could serve as valid controls or 
comparis^nFl' Second, an experimental design would not have been feasible 
to implement given the administrative structure and operating constraints 
of the district. Third, pursuant to earlier discussions with the NIE, it 
was understood that an experimental design was not a desideratum of the 
field test. 

One way that the field test evaluation dealt with the design problem 
was to compare results with national norms and baseline data. However, 
although norms and baseline data w*ere available for the assessment of 
students' achievement, there were no comparable data for measures of 
program Implementation and three of the student behavior variables (i.e., 
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academic per foriaance, student engaged time, and instructional overlap). 
There was no way of determining the status of these variables in the dis- 
trict in previous years; consequently, assessment of the impact of the 
approach on students was primarily limited to analysis of data gathered 
during the field -test year (1981-32). 

Interviews and observations with school and central office staff 
indicate that there were no major demographic changes during 1981-1982 
that would account for substantial Ranges in student achievement and that 
no new instructional programs were introduced during the field test year. 

Analytic issues 

The third set of limitations relates to analytic issues. Two 
important considerations are unit of analysis and the data analysis model. ^ 
Ideally^ the preferred unit of analysis for examining achievement gains is 
the^ classroom because Achievement Directed Leadership is intended to 
directly affect students in the classroom, ' and because any other unit of 
analysis above the classroom fails" to capture expected variation among 
teachers in their use of the approach. Hovjever, analyses were performed 

I 0 

primarily at. the school level and secondarily at the grade and/or district 
level because of the difficulty of obtaining implementation, process;, and . 
achievement data at the classroom level. For example, obtaining classroom 
level data, would -require gathering and matching achievement results from 
1980, 1981, and 1982 on e^ach student within a class. Massive manipulation 
of class lists and massive reorganization, of 1980 and 1981 data would be 
required to perform this match. Considerable effort was expended in attempt- 
ing to do this, but resulted in a rinal data set that included only about 
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40 percent of the 1981-1982 student: .population.. Statistical tests demon- 
strated that this group was not representative of the overall group* In' 
addition, implementation data at the elassroon level were suspect, as 
discussed later in this section. Consequently,^ the lowest level unit of 
analysis chosen for impact assessment was the grade level, The^ match 
between students moving from one grade level to the next, while not per- 
fect, was felt by district staff to be very good, and these data were much 
more easily obtainable than the classroom level data. The assumption made 
by the field test design is that the effects of grade level population 
changes were minimal. * 

Many authors have noted* limitations that must be considered when the 
norm-referenced evaluation model is used to analyze achievement effects ^ 
(e.g., Horst, Tallmadge, & Wood, 1975; Echternacht, 1978; Ozenne, 1978), 
While most researchers recognize these limitations, the model is still 
considered acceptable (e.g., U. S. Federal Register, 1979; Dermaline & 
""Rader, -1981; Maye, 1981; Tallmadge & Wood, 1980) and is. probably the most 
feasible and frequently-used evaluation paradigm for assessing achievement 
gains. The 'major limitation of the model relies to the assumption that 
the achievement status of ' the evaluation group remains constant? relative . 
to the norm group over the pre to posttest interval if no special treatment 
is provided. The validity of this assumption is unknown. Also, since^ the 
field test evaluation relied on available ai d accessible dat-a, results by 
grade from year to year were based on different test levels. This repre- 
sents a considerable problem for the interpretation of gains across years. 
The assumption that is made in the field test is that the norming pro- 
cedures used in test standardization result in comparable samples for each 



successive test^le^el. ' Since schools were uge(J as' sampling units for the 
nonning studies, this assumption seems reasonable. However, it must be 
gnized that the comparison groups 'do change, somewhat, from year to 
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year. 

Data Issues 



The final set of field test limitations has' to do with the Quality' of 
implementation and^ctassroom 'process' data. Although the ideal case, would ' 
have been for the BSC to collect" hard, objective data, economic and feasi- 
bility constraints precluded that option, 'consequently, all implementa- 
tion and classroom process data we^re based on self reports, The relia- 
bility of this informat'ion-.at the classroom level is somewhat questionable 
and may, in many instances, give spuriously high estimates. For this 
reason, data were" aggregated at 'the school level (for implementation) or 
grade level (for classroom process). In addition, due to the enormous 
'quantity of qualitative information, data measuring similar variables were 
aggregated into summary, indices. Although more confidence can be placed 
in these aggregated data, some fine details may be lost in the transla- 
■ tion. The aggregated information on program implementation and classroom 
processes seems to give fairly accurate, although .. somewhat global, indi- 
cators of activities and events at the field test schools. ^ 

In summary, it must be recognize4 that the field test was not 
desigvied to be an experimental study. Because of this , several 
assumptions were made which limited the study's ability to establish 
causal relationships between- program implementation and observed changes 
An -student behavior and achievement. The evaluation limitations do not,^^ 
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however, discount evidence gleaned from the field test. Rather, they 
emphasize the need for caution in interpreting results. Overall, the 
analysis of data collected during the field test provides a fairly 
accurate picture of the implementation and impact of Achievement Directed 
Leadership in the New Jersey School District. 
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CHAPTER FOUR 

FIELD TEST FINDINGS: NEW JERSEY SCHOOL DISTRICT 

'This chapter presents the field test findings in the New Jersey 
School District regarding installation and implementation of Achievement 
Directed Leadership as well as the outcomes related to the improvement 
program. Following a brief overview of the district, field test findings 
are discussed in relation to^ the five levels .described in the evaluation 
framework in Table 1 on page 19: BSC, district, principal, teacher, and 
student. The discussion of findings for each level indicates how the 
improvement process is expected to function at that level, what activities 
and outcomes actually occurred, and what conclusions can be drawn. The 
levels are interrelated in that results at the firs-t fotir levels suggest 
expectations for outcomes at the student level. A section between the 
teacher and student levels summarizes program installation and 
implementation, and details how expectations for student _level outcomes ' 
are derived. A concluding section summarizes field test findings in the 
New Jersey School District. 

\ ■ District Overview 

The New Jersey School District was selected as the site for the field 
test because of its comprehensive implemention plans for Achievement 
Directed Leadership. Although the BSC had been working in this district 
for three years, the previous work involved only. a few teachers in each of 
four elementary schools and focused on the development of class'room 
materials. 



In 1978-79 two schools (Schools A and H) became development sites; a 
few basic skills teachers at each school were trained in an early version 
of the time management materials and worked through the improvement cycle 
once, i.e., classes were observed, opportunities for improvement 
identified, and strategies selected and implemented one time. School H 
was eliminated as a development site for the following year (1979-80) due 
to its small size and relatively high student achievement scores. 

In 1979-80 a few basic skills teachers from School A, along with 
others from Schools C and G, were trained in time management; a few 
teachers from School A were also introduced to the management of instruc- 
tional content. Teachers at School A worked through the improvement cycle 
for time management twice during the year; teachers at Schools C and G 
worked through the cycle once. Implementation of content management was 
not attempted at School A. 

Tr> IQRO- Rl, select ed teachers fr om all three' schools wer^ trained in 

the management of content. These teachers attended "^to instructional 
overlap throughout the year, in addition to working through the 
improvement cycle once for time management. 

.In 1980 the district acquired a new superintendent, an experienced 
urban educator who had been an assistant superintendent of another urban 
school district in the state. He felt that the BSC approach had signifi- 
cant potential for improving basic skills instruction and achievement, and 
decided to implement the approach in grades K-8 district-wide during the 
1981-82 school year, the year of the field test. In addition, a former 



high school administrator with strong professional interest in curriculum 
and instruction was appointed assistant superintendent. These top level 
administrators played major roles in implementing and maintaining the 
improvement program throughout the field test. Other key central office 
staff involved in the field test included the superintendent's administra- 
tive assistant and several staff from the'Mistrict ' s newly created Depart- 
ment of Instruction (DOI) . The superintendent's administrative assistant 
had been a respected district principal and was responsible for the 
district's testing program, scheduling, and coimnunications . The DOI staff 
acted primarily as curriculum coordinators. 

This district consists of eight' elementary schools, one middle 
school, and a senior high school. All but the latter participated in the 
field test. Some characteristics of the nine field test schools are 
summarized in Table 3. Three field test elementary schools contained 
^ades K-6 and five contained grades K-7 . The middle school,, with grades 
7 and 8, was substantially larger than most of the elementary schools. ^ 
However, the elementary schools also varied in size, with the average 
daily Wollment ranging from 152, at School H, to 573 at School F. 

Although the district is relatively small, its student body had much 
in common with the student populations of many large urban areas: low 
socio-economic status, a high rate of student turnover (in 1979 more than 
5;0 percent of the students were leaving the system before completing high • 
school), low student achievement scores, and a high percentage of minority 
(90 percent) and English as Second Language students. In an effort to 
■'reverse the pattern of low achieyement, the new superintendent, at the 
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Table 3 



Description of Field Test Schools: New Jersey School District 



Characteristics 


School 
A 


School 
B 


School 
C 


School 
D 


School 
E 


School 
F 


School 
G 


School 
H 


School 
I 


Grades 'in School 


.K-6 


K-7 


'K-6 


K-7 


K-7 


K-7 


K-6 


K-7 


7-8 


Average Daily Enrollment 


377 


259 


431 


328 


323 


573 


317 


152 


662 


Average Daily Attendance 
Rate 


92% 


91% 


90% 


93% 


93% 


90% 


91% 


91% 


82% . 


Number of Classroom 
Teachers 


20 


17 


18 


18 


17 


32 


14 


8 


27 


Average Class Size 


17 


13 


19 


18 


20 


20 


20 


17 


25 


Average Years Experience 
of Teachers . ' 


Over 
10 


Over 
10 


Over 
10 


Over 
10 


Ovei; 
10 


Over 
10 


5-10 


Over 
10 


Over 




45 38 



time of his appointment, accepted the challenge of raising district test 
scores by the end of the 1982-83 school year. The strong interest of the 
school board in improving student achievement encouraged central office 
support of Achievement Directed Leadership, and probably accounts, in 
part, for the administration's insistence on full implementation of the 
program. The success in implementing the program as well as concomitant 
outcomes is discussed, by level, in tho sections that follow. 

The BSC Level 

In Achievement Directed Leadership, the functions of the BSC, or any 
external agent, are to plan and deliver orientation and initial training, 
and to provide follow-up technical assistance to practitioners to help 
them implement the approach. The goal of the BSC is to transfer the plan- 
ning, training, and implementation functions to local staff through a 
capacity-building approach. Responsibility for these functions should 
gradually shift from the BSC to the district. For example, regarding 
training, the BSC ideally provides district administrators with initial 
training in program concepts, models appropriate training skills, and 
directly assists in the district's' own initial training efforts. The 
expected outcome of these activities is the district's ability to ade- 
quately conduct its -own training for all staff. This section of the 
report -briefly describes how the BSC performed the functions of an ex- 
ternal agent as well as the outcomes of its efforts. 
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Orientation of Administrators • . ' 

Although several New Jersey School District administrators were 
Introduced to the time and content materials during the course of the 
earlier development process, all administrators scheduled to participate 
m full-scale Implementation received a basic orientation to program con- 
cepts and field test requirements during May and June 1981. The BSC held 
a series of orientation meetings with the superintendent and his Admlnls- 
tratlve Council. The council was comprised of four assistant super- _ 
mtendents, another special assistant to the superintendent, and the 
president of the principals; association.^ This series of meetings pro- 
vided the council with a systematic overview of the program. As a result 
of the meetings, the council endorsed the improvement approach 'as a dis- 
trict priority for the coming school year and plans were made for sched- 
uling and conducting orientation for all district principals.' Principal 
orientation, conducted by BSC staff late in June, consisted of an overview 
of the principal's role and responsibilities within the program. This 
overview was intended as a brief introduction i^rior to formal principal 
training scheduled later in the summer. 

Initiai- Training of Central Office Staff and P rincipals 

Initial training of central office staff and principals in the time 
and content variables was conducted by the BSC during a week in late 



^The Council was later expanded to include, the president of the 
local teachers' association. 
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sunimer devoted to general administrative training. Although most of the 
week was devoted to this training, part of- the agenda also included other 
district business. Consequently, the amount of. training time actually 
allocated to the program was somewhat less than originally anticipated and 
desired. Training workshops covered major program concepts and skills as 
well as implications for planning apid implementing the approach at the' 
individual school and classroom levels. 

All principals and assistant principals in the district as well as 
most of the central office staff attended the time and content workshops 
conducted by the BSC. Participants from the central office included the 
superintendent, the assistant superintendent for curriculum and instruc- 
tion, the assistant to the superintendent, and 11 members of the dis- 
trict's Department of Instruction (DOI) . A total of 15 hours was devote^d 
to training in management of the time and content variables, and to plan- 
ning for implementation. In addition to the two maj ot^jo^^kshops-p-resented. 
during the week of administrator training, BSC staff conducted a half-day 
planning session with all administrators ta prepare them for leading 
teacher orientation later in the fall. The primary BSC field person for 
the New Jersey School District felt that more administrator training would 
have been helpful. 

In general, training was well ' received by central office staff and 

principals. Reactions to the training are summarized in Table 4. The 

results show a very high degree bt satisfaction with all aspects of the 
training, with all workshop ratings in the "very good" to "superior" 
range. The highest ratings were given to the expertise of the training 
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Table 4 


> 


Summary of Participant Reactions to Training Workshops: New Jersey Schobl District 

— • : ■ ' 


Rating Category 


Content Management Workshop' 

Ce^nrrr:^^ Off-irp Staff Princioals (n=26) 


Time Management Workshop ' ^ 
Central Office Staff, Principals (n«26) 


Expertise of training staff 


A • C 

4 , 0 


4 7 

• 


i^uaxxLy Ox. pr tJotiiiLciLxuii/ i.Lctxiijj.Lg 


4.1 , 


4.4 


Quality of materials 


4.4 


4.5 


Relevance of content 


■ 4.1 




Likelihood that program will 
work in your situation 


4.0 . 


.3.9 


OVERALL RATING 


' 4.2 ■ 


4.4 » 




1 ... - - 


Scale 






5 = Superior 

4 = Very Good 

3 = Average 

2 = Below Average 

1 = Poor 







Data sources: post-training questionnaires 
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staff. Participants in the training sessions were also ■ asked to assess , 
their understanding of the major training objectives and to ijidicate h^^ 
confident they were in applying the concepts and techniques learned to a 
school situation. Detailed results are presented in Appendix E. Overall., 
administrators felt confident about their ability to perform the tasks 
needed to implement the improvement program. Most respondents to the 
training questionnaire did not report a need for further instruction and* 
assistance, although a few principals, who were interviewed after training 
expressed a desire for further assistance. Post-training interviews "indi- 
cated that training was perceived as adequate. Comments suggested that 
administrators felt slightly more confident about their conteut implemen- 
tation skills than their time implementation or teacher orientation 
skills. It was recognized that participation in the improvement effort 
represented a major change in principals' roles as instructional leaders. 
This caused some initial anxiety for a few principals, although most 
seemed reasonably comfortable with the modifications. Observations of the 
training sessions by BSC staff corroborated the post-training question- 
naire and interview findings. However, observations indicated that the 
level of program understanding and skills varied considerably across prin- 
cipals and that level of commitment to the approach ranged from medium to 
high following the initial tlraining. 

In summary, the initial training provided by the BSC was generally 
perceived by central office administrators and principal^ as successful in 
providing the knowledge, skills, and commitment needed to begin implemen- 
tation of the improvement approach in the New Jersey School District. 
Although, due to schedtiling problems, the amount of training was somewhat 
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less than that .suggested by the BSC, central office staff and principals 
were viewed by 'the BSC as sufficiently prepared to proceed with implemen- 
tation plans . 

Ongoing Support 

BSC technical assistance and implementation support should ideally be 
minimal after -Initial program orientation and training because the school 
district should assume major responsibility for program implementation. 
However, since the field test was limited to a one-year effort, the BSC 

"provided more hands-on assistance in the early stages than is suggeste4 by 
the Achievement Directed Leadership model. Assistance from the BSC field,, 
staff was available to the district as needed. Field staff were called on 
numerous times throughout the year to attend meetings, help solve 
implementatiQn -problems, plan and model activities, answer questions^ and 
reinforce understanding of major concepts. 

The BSC played a major role in supporting the district's leadership 
seminars. These seminars were a series of working sessions incorporated 

-within the superintendent's regular monthly meetings with principals. 
Since improvement was a district priority, seminars focused primarily on 
solving problems related to program implementation and on specific imple- 
mentation planning issues. For example, the sessions focused on issues 
such as establishment of an overall management plan for the program, 
development of formal plans for conducting teacher training, discussion of 
principal/teacher conferences, and development of institutionalization 
strategies for the next school year. The sessions also served as a 
primary vehicle for providing follox^-up training to principals. BSC staff 



52. 



mfet with the superintendent and appropriate district leadership prior to 
each seminar to decide on roles and r.esponsibilities for conducting the 
session. Initially, the BSC was a major contributor to the actual conduct 
of the seminars. However, the BSC involvement gradually diminished as the 
superintendent assumed moire responsibility for planning and Cisrhducting' the 
sessions. . By mid-year, the superintendent's role was, primary and that of 
the BSC secondary, as suggested by tlie improvement approach. Interviews 
with district administrator? indicated that support receiyed from "BSC 
Staff was perceived as adequate and beneficial. - ^ 

In addition to regular leadership seminars with principals ^ periodic 
meetings were held with the -superintendent and other central office staff 
so the BSC could "help them with implementation These meetings covered 

/ ! 

issues such as the principal/ teacher conference form, 'Students' daily 
success rate, and classroom.. impti^cOT^ra s"~b f Tlow^^andTo F~ va r ied^Iev e Is of 
students' entering achievement'; As with the principals' leadership 
seminars, the BSC role in conducting .these meetings gradually diminished 
as the superintendent assumed the major .leadership role. 

In addition to ongoing BSC support, the technical assistance services 
of another external agency, one of the state's regional Educational 
Improvement Centers (EICs) , were made available to administrators and 
teachers. The EIC was an important, partnex In the, development ^ of ^the ^ 
improvement approach in the New Jersey School District and EIC staff acted 

as back-up trainers durif^g the field test. | Assistance from the EIC was 

I 

available on request, and its staff provided support on a limited basis 

. 

during the. field test. - • 



Summary and' ConcluslDns . ' ^ < 

Overall, initial installation of the program in .the district Vas 
, , regarded as a success by central office staff. Training was well received 
and. administrators were confident that they .had acquired the necessary 
knowledge and .skills needed to successfully implement .all components of 
the program. Observations by .BSG^staff indicated that there was variation 
among principals in their skill development and initial commitment to the 
program. .This suggested, that ^th^e^quality of implementation would vary at 
the school level. Ongoing support was provided throughout the year to 
supplement initial , training and to assist in program planning, problem- 
solving, and maintenance. Follow-up . assistance by BSC staff, and leader- 
, ship training seminars were perceived ^y administrators as adequate and 
„ beneficial, 'as the school year progressed, district leadership, particu'- • 

larly the superintendent, gradually assj^med more responsibility for 

program maintenanc^e and implementation and the BSC role diminished as 
suggested by the program model. ^ . , 

y *" • The Distri ct Level 
^ » , •. ~ 

^Ms section provides an account of implementation of , Achievement' 
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Directed Letdership at the district leverand addresses the question of 
ho™d.^tet-«^-^t <.entral office staff perform^heir role-related 
functions: planning, training, and supervising. %^ous central office 
staff membets had major responsibilities for installing and implementing 
the improvement approach iu the New Jersey School District. Ideally, the 
BSC planning and training roles are assumed by these central -office staff 
members who are then responsible for performing these functions at the 



principal or individual school level. In addition, central office staff 
engage in participatory supervision with principals. In this process they 
monitor ongoing progress, and work with building administrators in plan- 
ning and directing implementation activities. Actual implementation of 
the approach by central office staff is briefly described below in terms 
of each of the three major district level functions. 

Planning 

Comprehensive planning by central office staff was a necessary ante- 
cedent to successful program, implementation. Implicit in all observed 
installation and implementation activities was a great deal of systematic 
planning. Planning specifically related to the field test began with the 
initial series of orientation meetings involving the BSC, tjie superinten- 
dent, and his administrative council. Planning activities continued 
throughout the year and concluded with the development of long-range plans 
for future school years. Each major decision regarding training anci 
implementation was preceded by comprehensive planning. For example, 
extensive planning was reqirired for such tasks as: - 

o preparing program budgets 

o allocating necessary resources c 

o scheduling training sessions 

o defining staff roles and responsibilities 

o determining participatory-supervision procedures 

o specifying training content 

o structurittg classroom observations 
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o developing curriculum guides 
o solving problems. 
•Central office staff used planning checklists developed by the BSC to 
guide their planning efforts (see Appendix F) . 

The superintendent was the key figure in the district's planning 
effort. Following the initial planning and training activities conducted 
during the summer, he developed a comprehensive "mission statement" for ; 
the district, outlining plans for improvement throughout the school year. 
This mission statement incorporated all key elements Of the improvement 
approach and became the basis for district level planning for the field 
test year. District goals were set, staff roles and responsibilities were 
explained, and procedures for reaching goals and assessing performance .... 
were detailed. The superintendent followed the plans he outlined in his ' 
mission statement through his continuing, active role In the improvement 
effort. Plans were systematically reviewed and updated through the series 
' of leadership seminars and other planning sessions described in the BSC 
level section of the report. As previously noted, although the BSC con- 
tributed heavily to the planning process in the early stages of program 
installation and implementation, the superintend^^ gradually assumed the 

primary planning role. 

several other central office staff assisted the superintendent in the 
■planning process. Whereas the superintendent was responsible for policy 
•planning, other administrators were given responsibility for planning__the_ 
specific ways that the policies would be carried out. For example, the 
assistant superintendent for curriculum and instruction was charged with.^^ 
planning' all training activities as well as planning a distric^t-wide 
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curriculum for basic skills. The assistant superintendent worked on these 
planning activities with the members of the Department of Instruction- 
(DOI) . The curriculum planning task was of central importance to content 
management. This planning effort resulted in the development of a 
curriculum mapping guide titled the Basic Skills Curriculum Guide (A 
Management Guide) for Reading. Langu a ge Arts and Mathematics Grades K-8 . 
which was analogous to the "School Year Planning Guide" specified by 
Achievement Directed Leadership. The mapping guide indicates those basic 
skills objectives included in the district's testing and assessment 
program for each grade level. The guide was intended as an overall plan 
for instructional content to be covered by each classroom teacher during 
the course of the school year. 

A DOI staff member was 'paired with each principal to assist in plan- 
ning at the school and classroom levels. In general, these staff worked 
closely with principals in planning for program implementation. District 
planning consisted of all preparations that were made to assure that 
principals, and to a lesser d^^ree, teachers, carried out their functions 
properly. However, the level of I^I involvement varied somewhat across 
schools. In addition, many DOI sta^reported that their level of 

involvement at individual schools lessen^as the year progressed. In 

X 

interviews administered at the end of the yeaK,DOI staff reported that 
they spent an average of 12 percent of their , time throughout the year per- 
torming all program related actxvities, includ ing planning, craxuxug, mia 



supervision. 
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Training 

The improvement approach suggests that the BSC train central office 
staff who, in turn, train principals who, in turn, train teachers in the 
management of the instructional variables. As previously noted, initial 
BSC training in the New Jersey School District was directed at both 
central office staff and principals. The superintendent, in conjunction 
with central office and BSC staff, provided follow-up training to princi- 
pals throughout the school year during leadership seminars. 

Although it was intended that principals assume primary responsi- 
bility for teacher training, DOT staff were paired with each principal an. 
assisted in teacher training activities. The extent of their involvement 
varied widely across schools depending, in large part, on each principal' 
knowledge, .kills, and commitment to the program. At some schools, DOT 
staff assumed the entire responsibility for teacher training yhile at 
others they played a relatively minor role. Specific outcomes of • teacher 
training for each school are discussed in the principal level section. 

Participatory Supervision 

The concept of participatory supervision at the district level 
requires that central office staff participate with principals in a 
variety of ways in order to strengthen the principals' problem-solving 
abilities and to reinforce their work with teachers in using the 



in the participatory supervision process and engaged in continuing 
dialogue with principals concerning program implementation throughout th 
school year.. A major formal vehicle for this dialogue was the district/ 
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principal conference.. Two conferences were conducted with each of the 
nine principals involved in the program. These conferences provided an 
opportunity for the superintendent and principal to discuss the critical 
dimensions of the school and the classroom and to review plans and 
activities. The superintendent indicated that the conferences were , 
beneficial both in helping him focus his own ideas on instruction and in 
assuring that principals were constantly aware of the instr|ictional issues 
"addressed by the improvement program. In addition to the d:^strict/ 
principal conferences, DOI staff assigned to each school moil^itored program 
implementation and worked with principals to solve problems W make sure 
that program operations were following plans. \ 

Summary and Conclusions 

Overall, central .office leadership in the improvement effcrt was very 
strong with the superintendent taking a very active rale in most aspects 
of planning and actual implementation. The superintendent, along with the 
assistant superintendent for curriculum and instruction, engaged in 
cooperative problem-solving with all DOI staff and principals at each of 
the test schools. Jointly, the superintendent, DOI staff, and other 
central office staff were responsible for district-wide planning and 
training. The DOI staff cootjerated with individual principals in planning 
and teacher training and monitored principals' progress throughout the 
year. 

Participation of central office staff in the improvement program 
represented a considerable change in their rol^es and responsibilities as 
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instructional leaders. Table 5 suranarizes information relating^ to percep- 
tions of these role changes. All central office staff and most principals 
felt that central office staff spent more time during the field test .year 
supporting basic skills instruction than in the previous year. Although 
many teachers agreed with this, several did not see a change in the antount 
of time since they had worked with DOI curriculum coordinators previously, 
although in substantially different roles. Principals and central office 
staff also agreed that the efforts of district level staff in supporting 
basic Skills instruction were more effective than in the previous year. 
Many teachers were not sure of the effectiveness of central office support 
since they had nqt seen the district's test results at the time of the 
survey and had minimal access to district-wide information. 

In summary, central office staff involvement in implementing the 
'program was very strong. The superintendent was very committed to the 
improvement approach and provided the extensive support necessary for its 
success. All district level planning and participatory supervision func- 
tions were effectively carried out in accordance with the program model. 
Although the BSC rather than the central office trained principals, a 
deviation from the model, all necessary training and support were avail- 
able and "seemed to be successful. 

Principal Level 

The basic functions of the principal in the improvement process are 
essentially the same as those of central office staff: planning, train- 
■ing.and supervising. Principals are a critical link in improving 
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Table 5 



Summary of Central Office Staff Implementation of ^ 
Achievement Directed Leadership: New Jersey School District 



Indicator 


"Respondent Group 


Perceived Change 


Perception of time central 
office staff spent support- 
ing basic skills instruc- 
tion in the schools, 
relative to prior year 


Central Orrice btarr 


Increase (all indicated increase 
in working directly with schools) 

/ 




Principals 


Increase (82% indicated increase 
in working with them) 




Teachers 


Uncertain (42% reported increase, 
others not sure) 


l^rercepLion or eiiecLive- 
Bhess of district in 
'supporting basic skills 

instruction, relative 

to prior year 


Central Office Staff 


Increase (all ■ indicated marked 
increase in effectiveness) 

• 




Principals 


Increase (all indicated increase 
in effectiveness) 




Teachers 


Uncertain (40% reported increase, 
others not sure) 



Data sources:' end-of-year questionnaires and interviews 
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schools. They are responsi'^le for, transl'ating global, dlstrlct-wid< 
policies into. action plans for guiding and monitoring instructional 
improvement at their school. Principals, perform the three major functions 
at their schools by working closely with central office staff and teachers 
in planning improvement activities, providing or arranging training 
experiences, and engaging in participatory supervision with teachers as 
they work together through the improvement cycle. .Principal activities 
are briefly described below in terms of the three major functions. 

Planning 

As was .the case for central office staff, comprehensive planning by 
principals was a necessary antecedent to successful program implementa^ 
tion. Systematic planning was implicit in ill observed activities at the 
principal level. Principals planned collaboratively with DOT staff and 
. teachers in scheduling and preparing for teacher orientation and training, 
structuring participatory supervision activities, and providing instruc- 
tional leadership related to the content and time variables. In general, 
principals followed the planning outlines for these activities as speci- 
fied in the Instructional Planning checklists provided by the BSC. These 
checklists are included in Appendix G. 

Much of the principals' planning efforts in the early part of the 
school year involved preparing for teacher orientation and training. 
Planning tasks included scheduling, determining faculty participants, 
setting agendas, handling logistics, and preparing for actual presenta- 
tions. During the school year, planning focused bti implementation issues 
concerning the two major focus variables (studeftt engaged time and . 
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instructional overlap) and on the participatory supervision 'process. 
Examples of these planning tasks were identifying prior learning data, 
assisting in curriculum matching activities, identifying appropriate 
instructional materials, identifying appropriate improvement strategies, 
arranging necessary inservice activities, developing schedules for 
classroom observations and principal/teacher conferences, and clarifying, 
procedures for conducting observations and conferences. 

Table 6 briefly summarizes^ principal level implementation at each 

■J 

school in terms of planning, training, supervisory activities, and 
attitudinal reaction to the program. Although all principals participated 
in the planning process to some degree, there was variation across schools 
in the amount of planning. All except one principal (School H) reported 
that they spent more time planning ^with teachers during the f ield ,test 
year than they did in previous years. However, teachers' perceptions of 
changes in their principal's planning role differed. At four schools 
(Schools B, CD, and G) they agreed with their principals that more time 
was devoted to planning, but teachers at two schools (Schools A and I) 
indicated that this was not the case. Teachers at other schools had 'mixed 
opinions about the relative amount of principal planning ^ In addition to 
findings presented in Table 6, all principals reported in end-of-year 
interviews that the^program helped them become more successful in helping 
teachers plan basic skills instruction. 

#= 

Training 

Achievement Directed Leadership suggests that principals deliver 
training, in the improvement approach and the relevant instructional 
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Table 6 




Sammary of Principal 1™?!^™^""^°". , District 
of Achievement Directed Leadership: New Jersey School, District 



Principal Level Variables 
and Data Source 



Did principal spend more time 
planning with teachers than 
in previous years? 
(principal questionnaire) 



Did teachers feel that princi- 
pals spent more, time planning 
with them? 

(teacher questionnaire) 



Did principal conduct 
inservice training relating 
to program? (observations) 




Average number of observa- 
tions per teacher 
(teacher questionnaire) 



Average number of. conferences 
per teacher 

(teacher questionnaire) 



Average number of conferences 
per teacher (principal, 
conference forms) 



Did principal feel that over- 
all effectiveness of school's 
basic skills instruction 
increased? 

(principal questionnaire) 



Did teachers feel the school s 
effectiveness in providing 
basic skills instruction 
increas^ed? 

(teacher questionnaire) 



Attitudinal reaction of 
principals to program 
(observations, interviews, 
and principal questionnaire) 



4. « Yes from a majority bf ^respondents; 
- o no from a majority of respondents; 
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variables to teachers at their respective schools. As indicated in Table 
6, this paradigm was employed at six of the nine schools participating in 

the field 'test. At , the other schools, DOI staff assumed major responsi- 

t 

bility for delivering teacher training. Orientation and' training activi- 
ties were conducted at the same time at the nine schools, A general 
orientation was given during the first day of school. Content training 
was delivered in a three-hour inservlce session later in September , while 
time training was delivered (n a two-hour session in early October. All 
teachers participated in the training workshops. BSC staff observed 
training sessions at a sample of schools. These observers reported that 
although training generally followed program specifications, the quality 
of the presentations and the attentiveness of the audiences varied across 
schools. Observers also noted that the content sessions seemed to go more 
smoothly than the time sessions at most schools. 

All participating teachers completed post-training questionnaires 
where they indicated their reactions to the training workshops and 

assessed their acquired knowledge and skills related to program concepts. 

I. 

Results are briefly summarized in Table 7. Overall reactions to the 
training were 'Very^ positive at most schools. The ratings for expectations 
regarding the likelihood of program success were somewhat lower than the 
ratings for overall reaction to the training. In fact, teachers at three 
schools (Schools B, C, and H) were doubtful 'that the time component of the 
program would work in ttieir schools. BSC observations indicated that 
there were some problems with training workshops at these schools. * Oveir^ 
all, teachers at all schools felt confident that they Md acquired the 
knowledge and skills needed to succ^sfully im^ilement the improvement 
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Table 7 



Summary of Teacher Training Results: New Jersey School District 





School 


All 
Teachers 




A 


B 


C 


D " 


E , 


F 


G 


H 




CONTENT 






















Total number of teachers^ 


25 


17 


23 


i 


24 




21 


8 







Overall reaction to training^ 


3.9 


4.1 


3.7 





4.4' 


' 4.0 


3.7 


4.4 





4.0 


Program expectations 


3.5 


3.6 


3.1 





3.8 


3.8 


3.4 


3.3 ' 




3.5 


Overall assessment of knowledge/ 
skill development 


4.0 


4.3^ 


4.1 


— ~ 


4.3 


4.1^ 


4.5 


4.7^ 




4.3 


TIME 

/ 

Total number of teachers'^ 






















25 


14^' 


14 - 


20 


24 


30 


18 


14^ 




145 


Overall reaction to traiaing^ 


3.9 


3.5 


2.8 


3.8 


4.4 


4.0 


4.1 


3.5 




3.9 


Program expectations 


3.0 


2.7 


2.1 


3.2 


3.8 


3.4 


3.8 


2'. 7 




3.3 


Overall assessment of knowledge/ 
skill development 


3.8 


4.3 


3.5 


3.7 


4.0 


3.9 


4.5 


4.3 




3.9 



CO 



Scales: Reaction and Expectations 



5 = Superior 

4 = Very Good 

3 = Average 

2 - Below Average 

1 = Poor 



bo 



Self -Assessments 

5 = On my own / 

4 = With further stuSy (content), or with planning 
3 = With some assistance/ 

7 - Only with further instruction and assistance 

1 = Even with further instruction, may not be aj)le to do 



a Results not available for School D (content) and School L (time and content) 
b Teacher |^n"s vary somewhat, by question 

c Summary of reactions to staff, presentation, materials, and content relevance 
d Assessments based on five of ten items; other items omitted 
e Time training combined for Schools B. and H 
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s: Post-training questionnaires 




process, although the ratings for time were lower than those for content. 
Interviews -with a sample of teachers at the end of the year indicated that 
program implementation became easier as they worked through the improve-^ 
ment cycle and became more familiar with the practical application of 
program concepts. In summary, teacher training was generally regarded as 
successful in providing the prerequisite knowledge and skills needed for/ 
program implementation at the classroom level. However, some problems 
were observed in terms of the apparent level of teacher commitment and 
expectations at certain schools. This suggested that variations in sub- 
sequent program implementatir n might be -expected. 

Participatory Supervision 

Participatory supervision at the principal level is analogous to that 
described at the district level, in that principals .monitor teachers' 
performance related to program functions and engage in continuing dialogue 
with teachers on instructional improvement. Principals observe basic 
skills classroom instruction, work with teachers in identifying opportuni- 
ties for instrnctional improvement in the classroom, and assist teachers 
in the selee<tion, implementation, and evaluation of improvement strate- 
gies. The^^rimary vehicles for this supervision are structured classroom 
observations and principal/ teacher conferences. 

Table 6 on page 56 also presents information on the average numbers 
of classroom observations and principal/teacher conferences reported for 
each schobl. Most tea^'ctiers reported that principals observed Cheir 
classes approximately six times ' and followed-up with an average of about 
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three conferences These results indicate that district plans for 
participatory supervision were carried out at virtually all schools, 
interviews with principals and teachers regarding the quality of partici- 
patory supervision suggest that the activities were beneficial and 
fostered improvement in basic skills instruction. Principal and teacher 
questionnaire data on improvement of basic skills instruction, as summar- 
ized in Table 6, corroborate the interview comments, although teachers at - 
five schools were uncertain whether improvements had occurred. Almost all 
teachers and principals welcomed the opportunity to talk together on a 
one-on-one basis about classroom instruction, and several principals 
reported that they were glad that the structured .observations "forced" 

them to visit classrooms • 

Also included in Table 6 is a summary of each -principal' s overall 
attitudinal reaction to the program. Four principals had positive 
attitudes., three had neutral attitudes, and two had negative attitudes, 
in the two schools (Schools B and H) where the principal had a negative 
attitude, the tr/I^ing of teachers was conducted by central office staff, 
not by the principal. The principal's attitude did not appear to 
significantly influence the number of principal/ teacher conferences or 
classroom observations conducted by the principal. 



^TVrn .sources of self-report data were available regarding the 
number^f conf^rLcL! The accurate total probably lies between the two 
reported figures • 
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Summary and Conclusions 

In general, implementation of Achievement Directed Leadership at the 
principal level proceeded according to plans although some variation among 
schools was observed. Systematic planning pteceded all principal level 
activities. Teacher training was generally regarded as successful, and 
principals carried out their supervisory activities as planned. 
Principals felt that the approach was successful in improving their 
school's effectiveness in providing basic skills instruction, but teachers 
were not sure since they had not seen students' test results. 

Principals varied in their attitudinal reactions to the program. ^ 
Their attitude appeared to influence their participation in teacher train- 
ing, but not the number of principal/ teacher conferences and classroom 
observations tihey conducted. 

Variations in principal level implementation for each school are 
[ ■ .3 
summarized in liable 8- in terms of summary indices for each of four 

important principal level variables: attitude toward program, classroom 

observations, principal/teacher conferences, and training results. The 

indices indicate f,or each variable whether level of implementation can be 

described as high, \medium, or low with_respect to program expectations. 

The overall princip^ implementation index summarizes the indices of the 

four variables and reflects the variation between schools in the degree of 

principal level impleiijentation. As indicated. in Table 8, implementation 

at the principal level\was rated high at one school (School E) , medium at 

\ ■ ' ■ 

five schools (Schools A> D, F, G, and H) , and low at three schools 



/ ^Specific rules for 'assigning indices are discussed in Appendix D. 



Table 8 



Summary Indices^ for Principal Implementation . 
of Achievement Directed Leadership: New Jersey School District 



f 


Indices^ for 
TD>.-J«^-J,^o1 T.o^rol variables ^ 






School 


Attitude 

Towards 

Program 


Classroom 
Observations 


Jl 
It 

Principal/ 

Teacher 

Conferences 


Results 


Sum 
of 
Indices 


Overall 
Principal 
Implementation 
Index 


A 


3 


3 


J. 


2 


9 


Med 


B 


1 


2 


1 


2 


6 


Low 


C 


2 


2 


2 


1 


7 


Low 


D 


2 


2 


2 


2 


8 


Med 


E 


3 


2 


2 


3 


10 


High 


F 


2 


3 


2 


2 


9 


Med 


G 


3 


2 


1 


3 


9 


Med 


H 


1 


3 


3 


2 


9 


Med 


I 


3 


1 


1 


2^^ 


7 


Low 



^Indices describe whether implementation is considered relatively high (index - -3' 
medium (index ' ••2-), or low (index = "l") • Specific rules for assigning each 
■ index are discussed in Appendix D. 

^Data sources: end-of-year questionnaires, surveys, and principal/teacher 
conference forms. 

'^Data not available, assigned average value. 
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(Schools B, C, and I). When these measures are combined with teacher 
implementation indices, an overall school implementation index is derived 
which suggests expectations regarding student achievement. Teacher and 
school implementation indices are presented in subsequent sections of the 
report. 

Teacher Level 

Implementation of Achievement Directed Leadership at the teacher 
level involves using the improvement cycle to guide planning,, classroom 
management, and instruction. Teachers are expected to„use the improvement 
cycle- to set instructional goals, identify opportunities for improvement, 
and make necessary changes in instructional procedures. Ih the improve- 
ment cycle, teachers attend to several targeted classroom variables: 
prior learning, student engaged time, instructional overlap, and academic 
performance. The following section briefly describes use of the improve- 
ment cycle during the field test by basic skills teachers in the New 
Jersey School District. 

Teachers Use of the Improvement Cycle 

The degree to which teachers actually implemented components of the 
improven^ent approach in their classrooms varied widely across schools. 
Table 9 summarizes information reported by teachers concerning their use 
• of Achievement Directed Leadership during the year, as well as their 

perceptions regarding its effectiveness and their general reactions to. it. 
All data are based on responses to the Teacher Questionnaire administered 
to participating teachers at the end of the field test. 
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Table 9 



Summary of Teacher Implementation 
of Achievement Directed Leadership; New Jersey School District 









S 


C H C 


) 0 L 


— \r=— 








District 
Total 


Teacher Level Variables ^- 
percent of teachers who: 


A 
N=13 


B 

N-10 


C 

N=14 


D 

N=ll 


E 

N=16 


F 

N=23 


G 

N=16 


H 
N=6„ 


I 

N=18 


Used research to set goals and 
improve instruction 


81 


70 


57 


95 


64 


61 


67 


45 


79 


69 


Made improvements regarding 
targeted instructional variables 


78 


34 


40 


78 


67 


63 


97 


40 


76 


64 


Riirrpoded in obtainine; 
appropriate levels of targeted 
instructional variables 


72 


67 


58 


.51 


53 


61 


72 


46 


64 


61 • 


Made changes in planning, 
instructional management, and 
teaching techniques 


52 


53 


33 


55 


47 


55 


72 


33 


42 


49 


Judged ovm classroom instructign 
in^a^ic skills more effective 
this year 


77 


40 


43 


55 


43 


64 


60 


25 


59 


54 
























Attitude toward program, 
end-of-year (mean rating, scale 
1 - 5^ ) 


3.2 


2.5 


2.5 


3.0 


2.9 


3.0 


3.5 


1.6 


3.3 


2.9 

J — ^ — 4 



"7^3 l^ata source: end-of-year questionnaire; the figures in the table indicate the percentages of teachers who 

H , " ^ ^ , responded ^Ves" to questions that correspond to the teacher level variables (except for the 
D Attitude Scale ^ ^ ^ . attitudinal results) 

1 - very negative 4 — positive . 

2 - negative 5 - very positive • - . 



3 - ncMitn^'U 



er|c 






The majority of teachers (69%) reported that they used research !' 
findings as well as classroom data to set improvement goals fot their 
classrooms and to improve their teaching. This ranged from a Ibw of 45 
percent in School H to a high of 95 percent in School D. SixtyAfour 
percent of the teachers, ranging, from a low of 34 percent>,in School B to a 
high of 97 percent in School G, reported that they made improvements 
regarding the targeted instructional variabj.es. Sixty-one percent, 
ranging ftom a low of 46 percent in School H to a high of 72 percent in 
Schools A and G, perceived that they were successful in obtaining appro- . 
priate levels of these variables in theip classrooms. Mote specifically, 
teachers indicated that they attended to prior learning , instructional 
overlap, student engaged time (SET), and, to a lesser decree, academic 
performance. Ex;tmples of activities ^ related to each of tjhese variables 
are noted below. 

To obtain T^io^^ learning data, a standardized diagnoktic/prescriptive 
test was administered to all students in mid-September. Results for each 
student were provided to teachers on each skill tested. Im addition, 
teachers were given students ' 1981 summary scores on the California 
Achievement Test (CAT), which indicated each student's genetal level of, 
achievement. Most teachers reported that they used these daita in planning 
classroom instruction, although many felt that they were not as successful 
in attending to prior learning. as they were for SET and instructional 
overlap. Although teachers were introduced to the importance! of prior 
learning, training was not as comprehensive as it was for theWo vari- 



ables mentioned above. 
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All teachers were given the curriculum mapping guide developed by the 
district to indicate the overlap between instructional objectives and test 
content-. In previous years the concept of "mastery learning" was empha- 
sized throughout the district. However, during the field test year 
instructional overlap was to receive equal emphasis. District policies 
directed teachers to cover all objectives during the course of the year/ 
Teachers translated objectives listed in the district's curriculum mapping 
guide into their own "Quarterly Topic Flans." These plans took students' 
strengths and weaknesses into account and represented teachers' instruc- 
tional plans for each quarter. Each teacher's plan was unique. Teachers 
discussed progress with respect tq their plans with principals during 
their principal/ teacher conferences. Most teachers reported that they 
were successful in achieving very high .overlap with test objectives during 

the course of the field test. 

As noted in the discussion of the principal level, teachers reported 
that their classes were observed an average of five to eight t'imes during 
the year. These observations 'consisted of a serlT^&j of scans (i.e., 1-3 
minutes) during which each student's behavior was classified as engaged or 
unengaged. Teachers were thus made aware of students' use of instruc- 
tional time. Teachers discussed results with principals during confer- 
ences and, when necessary, they designed strategies for improving the 
daily average SET for their class. Most teachers reported that they made 
instructional improvements in their classrooms to increase SET and most 

c 

felt that they were successful in achieving high SET. 

The fourth variable targeted for improvement, academic performance, 
involved attending to students' daily success, mastery of content, and 



review of content. At the time of the field test, the BSC had not 
developed formal procedures for training educators in this variable, 
although teachers were made aware of its significance. Although teachers 
at many schools reported that they made some classroom improvements in 
these areas, they did not feel that they were as successful in dealing 
with academic performance as they were . for other target variables. 

According to the program model, teachers can affect the levels of the 
instructional variables in their classrooms by making changes in instruc- 
tional processes, such as instructional planning, classroom management, 
and delivery of instruction. Althoug-b many teachers (49%) reported that 
they made changes in these processes, particularly with regard to 
planning, overall changes reported at several schools (Schools C and H) 
were relatively minor. For the most part^ teachers indicated that they 
did not actually change their teaching techniques. 

Teachers' perceptions of program success in improving the effective- 
ness of basic skills instruction in their own classrooms varied widely 
across schools. A majority of teachers (54%) felt that instruction was 
more effective during thei field test year than in other years. This 
ranged from a low of 25 percent in School H to a high of 77 percent in 
School a: Many teachers indicated that their basic skills instruction ^as 
effective in prior' years, as well as in the field test year. Finally, 
many teachers were not sure if instruction was more effective since they 
had not seen students' test results. 

Teachers at most schools reported neutral attitudes toward the 
program. Reactions were negative at a few schools. While many teachers 

• * 



felt that the program wks valuable, many also indicated that it pquired a 
great deal of additional work. , ' 

Summary and Conclusions 

Overall, program implementation at the teacher level was regarded as 
successful in terms of expected instructional activities in the classroom. 
Field test teachers, in general, reported that they used research infor- 
mation to 'guide the instructional improvement process and indicated that 
they attended to targeted instructional variables. A majority of them 
seemed to adequately implement the improvement approach. _ » 

. However, considerable variation in the reported degree of teacher 
implementation was observed between schools. Teachers^ reports of their 
activities indicated that overall implementation was relatively high in 
relation to the program model at some schools, but relativ<^ly low at other 
schools. Table 10 presents information on teacher implementation in terms 
of summary indices for each of the data categories described in Table 9 
and in the preceding narrative. Specif ic .rules for deriving the summary 
indices are discussed in Appendix D. The ovl^aii teacher implementation 
index suggests that teachers engaged in the improvement process to a high 
degree at two. schools (Schools A and G) , to. a medium degree at four 
schools (Schools B, D, F, and I), 'and to a low degree at three schools . 

(Schools C, E, and H) . In the next section, these measures are combined 

I 

with principal implementation measures to produce overall school 
implementation indices which suggest expectations fbr student achievement. 
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Tab3i!e 10 



Achievement 



Summary of Indices^ for Teacher Implementation of 
ement Directed Leadership: New Jersey School District 



1 . Indices for Teacher Level Va 


riables ^ 


^ 




Overall 
Teacher 
Implemen- 
tation 
Index 


i 

School S 


; Research 
Use 


' Instruc- 
tional 
Improve- c 
ments 


■Success 
in Imple- 
mentation 


Changes in 

Teaching 

Behavior 


Improved 
Classroom 

HiTl cCl. XyC 

ness , 


Attitudes 

TfSwflTd 

Program 


Sum of 
Indices 


^> A 


3 


3 


3 . 


2 


3 


2 


16 


High 


! B 




1 


2 


2 


1 


1 


10 


Med 


C 


2 


1 


2 


1 


1 


1 


8 


Low 


D 


3 


3 


2 


2 


2 


2 


14 


Med 


E 


2 


2 


2 


1 


1 


1 


9 


Low 




. 2 


2 . 


2 


2 


2 


2 


- 12 


Med 


It 

G 


2 


3 


• 

3 


3 


2 


'3 


16 


kgh , 1 


H 


1 


1 " 


1 


1 


1 


1 


6 


Low 


I 


3 


3 


2 


1 


2 


■ 2 


13 


Med 



^ Indices- describe whether implementation is considered relatively high (index = "3"), 
medium .'(index = "2"), or low (index =..J'L") . Specific rules for assigning each 
;index are discussed in Appendix D. 

ti-Data S.ource: End-of-year questionnaire. 
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Summary of Program. Installation and Imp lementation 
In the New Jersey District 

The New Jersey District agreed to full-scale program. Implementation, 

i 

and was, therefore, the focus of the field test evaluation. To a large 
extent, the success of the field test was dependent 6n the degree to whi|:h 
the district actually followed through with Its plans for comprehensive 
Implementation. This section summarizes the actual degree of installation 
and Implementation as reported for the four preceding levels. 

; Initial Installation of the program at the district level was 
generally regarded as a success by central office staff- and principals. 
Training of central office staff and principals was well received, and 
these administrators were confident that they had acquired. thfe knowledge 
and skills needed to, successfully Implement all components of the program. 
Leadership training seminars and ongoing follow-up support by BSC staff 
were also perceived'as being adequate and beneficial. This suppo.rt, while 
considerable in early stages of the field test, gradually diminished as 
district leadership, assumed the major roU in- guiding and maintaining the 

improvement effort. 

Central office Involvement in the improvement process was very 
strong. The superintendent was highly committed to the improvement 
approach and provided the support necessary for its success. Some central 
office personnel (i.e., DOI staff) worked very closely with each of the 
elementary school! principals in planning, training, supervising, and 
providing technical assistance to teachers. Central office staff, 
themselves, ^perceived that their role had changed from th'e previous year 
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and that they were more supportive of basic skills instruction at the 

school level than in the previous year. . ' - 

• /' 

■ / •. / 
/ Implementation at /the school and classroom levels generally followed 

' .. ' • / . ■ 

the plan outlined by the program model. For the^raost part^ principals 

performed planning, training, and supervising activities /as expected.* 



However, some variations across schools were observed ^Ith respect to the 
Actual degree of implementation and the apparent level of comodditment to 
the improvement effort. Likewise, teacher activities, in general, 
followed the model. Teachers- reported that they used research to guide 
instructional improvement, attended to targeted classroom variables, and 
perceived that they were successful in reaching and maintaining appro- 
priate levels of the targeted variables in their classrooms. However, as 
with principals, levels of teacher implementation atid apparent commitment 
to the approach varied widely across schools^ 

Information on levels of principal and teacher implementation of the 
improvement approach was categorized in terms of summary indices as 
described in preceding sections and in Appendioc D. These summary indices 
describe whether principal or teacher implementaticJn was high, medium, or 
low with respect to, expectations suggested by the program model. An 
overall school implementation index was derived -by combining the principal 
and teacher indices. , The specific variables for each level^ encompassed by 
the overall school index are: 

o Principal Level o 

- attitude toward program 

- number of classroom observations conducted 
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- number of principal/teacher conferences conducted 

» 

- training outcomes (participant reactions and self- 
assessments of knowledge and skills). . 

o Teacher. Level 

- use of research to guide instructional improvement 

- improvement in classroom instruction in terms of program 
focus variables (relative to prior years) 

- changes in instructional planning, classroom management, 
and actual classroom instruction 

- perceived success in implementation 

- perceived effectiveness of classroom instruction 

- attitude toward program. ■ 

High indices for any of the above variables indicate that the teacher or 
principal behavior or attitude is at a level which closely approximates an 
ideal implementation of the program with respact to that particular 
variable. Low indices, on the other hand, indicate that implementation is 
well below desired levels. Medium indices fall between these extremes; 
they suggest that implementation, although not perfect, is generally 
satisfactory but not of sufficient magnitude to warrant a high rating. As 
Appendix D indicates, decision rules for the assignment of indices were 
somewhat arbitrary. However, they are useful as gross descriptors of 
relative levels of implementation. 

Table 11 presents the overall summary indices for principal and 
teacher implementation (also see Tables 6, 8, 9 and 10) as well as an 
pverall schoolwide implementation index for each school. The overall 
schoolwide index represents the sum of the four principal level indices 
and the six ^eacher level indices, and is, accordingly, influenced more by 
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Table 11 



Summary of Schoolwide Implementation , ^, , . ^ 
of Achievement Directed Leadership: New Jersey School Distrrct 









S 


C H 


0 0 


L 








Index 


A 


B 


C 


D 


E 


F 


G 


H 


I 


Principal Level 
Implementat ion 


Med 


Low 


Low ^ 


Med 


High 


Med 


Med 


Med 


Low 


Teacher Level 
Implementation 


High 


Med 


L0V7 


Med 


Low 


Med 


High 


Low 


Med 


Overall 
Schqolwide 
Imp 1 emen t a t io n 


High 


Low 


Low 


Med 


Med 


Med 


High 


Low 


Med 



'see Appendix D for discussion of procedures for assigning indices. 
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teacher behaviors than by principal behaviors. The rationale for this 
weighting schema is that classroom level implementation will, most likely, 
have a greater effect pn student achievement and that teachei indices are 
more reliable given the larger samples. 

The results suggest that level of implementation varied widely across 
schools with two schools (Schools A and G) having "high" indices, four 
schools "medium" indices (Schools D, E, F, and I), and three schools "low" 
indices (Schools B, C, and H) . The source of the variation is not readily 
apparent. Some variation occurs as the improvement process flows through 
each successive level of the hierarchy. However, it should be noted that 
the degree of teacher level implementation did not necessarily follow from 
-the degree of principal level implementation. Prior involvement also may 
have affected implementation during the field test year* Both "high" 
implementation schools had participated in the development of Achievement 
Directed Leadership during p^vious school years, although notion a 
schoolwide basis. However, the other two schools that participated in 
development (Schools C and H) were rated as "low" implementation schools. 

One probable major source of variation .is level of commitment of 
principals .nd teachers, an area that was not formally assessed during the 
field test. Scriven (1973) suggested that degree of motivation and 
commitment may have a large influence on implementation of educational 
innovations. Following this line of thought, Lipe and Haveman (1977) 
posited four levels of commitment in field test situations which may help 
to describe the variations in implementation between schools in the New 

■ - , \ ' 
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Jersey School District. They categorize field test participants as: (1) 
agressively resistant, (2) passively resistant, (3) personally committed 
or (4) group committed. 

Each of these four levels of commitment was observed by BSC staff in 
one or more of the field test schools. Where the principal and teachers 
were either actively or passively resistant, school implementation seemed 
to be low. Where some individuals, but not the entire group, were 
committed, school implementation appeared to be medium. Where there was " 
group commitment, school implementation was regarded as high. Thus, level 
of commitment seemed to be a major factor in implementation"behavior . 

In conclusion, the degree of implementation of Achievement.- Directed 
Leadership can be considered as moderate in the New Jersey School 
District. All schools attended to the time and content variables and _ 
implemented the process to some degree, although some schools were more 
committed than others to the overall approach according to BSC obser- 
vations. This variation in commitment seemed to account for variations in 
overall levels of implementation among schools. 



Student Level 



The hypothesis of the field test was that installation and 
implementation of Achievement D^-rected Leadership at the four levels 
described above would affect students' classroom behaviors and academic 
achievement. The acrval degree of implementation at each, school would 
logically affect the degree to which student outcomes at each school are 
influenced. Program effects at the student level are described below in 
terms of student behaviors/classroom processes and basic skills 



achievement* A final subsection describes the relationship between 
student outcomes and level of installation/implementation. 

Student Behavior/Cliassroom Process Outcomes 

As indicated in the discussion of program implementation, New Jersey 
School District principals and teachers generally addressed all af the 
classroom focus variables, although to varying degrees. Targeted class- 
room process variables include prior learning, student engaged time, 
instructional overlap, and academic performance. Outcomes for each of the 
variables are discussed below, although data for some variables were not 
sufficient to allow conclusions to be drawn. 

Prior learning. Teachers indicated that they attended to students' 
prior learning strengths and weaknesses in planning instruction and 
attempted to remediate deficiencies throughout the school year. Detailed 
diagnostic results for each student indicating mastery or n^nmastery of 
specific skills were given to teachers at the beginning of the year. How- 
ever, a comprehensive analysis of these detailed diagnostic tests is 
beyond the scope of this report. The only conclusions that can be made 
regarding prior learning are that adequate information,, on prior learning 
was available to teachers and that they reported that they used this 
information* , " 

Student Engaged Time* In training in the management of students' 
classroom titne, it is suggested that if students spend a s-pecified amount 
of engaged time in basic skills instruction during >the school year, 
corresponding growth in student achievement may be expected* These expec- 
tations are based on data from. the Stallings and Kaskowitz (1974) study of 



Follow Through eligible classrooms and the Beginning Teacher Evaluation 
Study (Fisher, Filby, Marliave, Cahen, Dishaw, Moore, & Berliner, 1978). 

Based on the actual level of student engaged time (SET), one would predict 

' 4 
students' test scores to exceed expected achievement levels, to fall 

below expected levels, or to equal expected levels. 

Data on student engaged time were collected during observations in 
each basic skills classroom in the New Jersey School District (see 
principal level, p. 59). Teachers compared their classroom data to data 
from the research base to establish SET goals, and their SET data from 
subsequent observations gave the empirical feedback on how well they were 
achieving their goals. Goals were typically set in the SET range where 
student achievement would be in the above expected range.- 

Table 12 indicates the average daily SET, computed across the 
observations (approximately six total) , for each school in reading and 
mathematics. In ad'dition, the percent of classes in each expected achieve- 
ment range is presented and a school summary index is provided which is 
an average of all classroom SET indices. Since research data can not be 
extrapolated to seventh and eighth grades, indices for these grades were 
not computed, and, therefore, are not part of the school summary indices. 
This summary index provides a general indication of what would be expected 
in terms' of student achievement growth given the average SET levels. 
Overall, observed SET was high across the district. A majority of 



Expected levels of achievement are defined as typical rates of 
growth relative to national norms, i.e., no gain in NCE scores from year 
to year. 
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Table 12 

Summary of Student Engaged Tinie (SET): New Jersey School District 

Reading^ 



School 


INO • OX 

Classes 


7 in 

Below 

Range 


% in 

Expected 
Ranpe 


% in 

Above 

Range 


Average 
SET 


Indexb 


A .; 


15 


13% 


33% 


53% 


1 9ft 

izo 


3 


B 


12 


67% 


0 


33% 


on 


■ 9 


C 


16 


31% 


31% 


38% 


lUo 


9 


D 


12 


33% 


50% 


17% 


yj 


9 


E 


16 


6% 


19% 


75% 


132 


3 


F 


Z J 


4% 


9% 


87% 


126 


3 


G 


12 


0 


17% 


83% 


125 


3 


n 


7 


0 


43% 


57% 


120 


0 


T 


/'I 

ci-5; 








(56) 


- 


Total 


1 1 '5 


18% 


23% 


59% 


116 


3 






a 

Mathematics 






School 


No . or 

Classes 


% in 

Below 

Range 


% in 

Expected 
Range 


% in 

Above 

Range 


Average 
SET 
(Mm . / Day; 


SET 
Index 


A 


■ 14 


7% 


57% 


36% 


46 


J 


B 0 


12 


0 


17% 


83% 


58 


o 
3 


C 


16 


0 




63% 


50 


Q 


- U 


12 


8% 


17% 


75% 


55 


3' -c;/. 


E \ 


16 


31% 




56% 


46 


2 


F 


23 


4% 


17% 


78% 


57 


3 


G 


12 


0 


33% 


67% 


55 


' 3 


H 


7 


0 


0 


100% 


66 


3 


I 


. flO) 








r50> 




• Total 


112 


7% 


25% ' 


68% 


54 


3 



^SET Index not appropriate for grades 7 & 8; averages for these 
grades not included in totals ' 

b * \ ' ' . 

Index, Key <^ ■ . 

1 = Below expected rate. 

2 = A!: expected rate. 

3 = Above expected rate. 

overall school indices are an average of all ^^1-^^°°- ^^^/^Jj'^^^; 
^nd, therefore, do not always reflect the average daily SET for the 

school. . '~ 
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el^ementary school classes in both reading and mathematics reported SET 
levels which indicated that student growth should exceed expected levels, 
although three schools (Schools B, C, ^4nd D) had reading SET levels in the 
expected achievement range , and ong^ school (School E) had mathematics SET 
levels in the same range. In Summary, teacher goals for maintaiuing high 
levels of SET were achieved in most cla-s^es throughout the district. 
Assuming that the average SET from all observations for each class was 
representative of the entire school year, one would expect that student 
achievement gains would be correspondingly high. . - 

Instructional Overlap . Instructional overlap is operationally 
defined as the perient of test objectives covered by teachers .in actual 
classroom instruction. Research has demonstrated that students' achieve- 
ment relative to expected growth is strongly influenced by the degree of 
instructional overlap with the test (Brady, Clinton, Sweeney, Peterson, & 
Poynor, 1977). Depending on the amount of overlap, growth in student 
achievement may be expected to exceed, equal, or fall below predicted- 
levels. As was the case with SET, indices can be assigned for each qf 
these three ranges of expected growth. 

Data on amount of instructional overlap were collected from partici- 
pating teachers on instructional content forms that were keyed to the 
California Achievement Tests (CAT) at each grade level (see Appendix C for 
sample form). Table 13 indicates the average amount of instructional • 
overlap per classroom at each school. In addition, the percent of classes 
in .each, expected achievement range is presented as well as an overall 
summary index which is an average of the classroom indices. Clearly, the 



Table 13 

Summary of Instructional Overlap (10); New J6raey School District 



Reading^ 



School 


No. of 
Classes 


% in 
Below 


% in 

Expected 


% in 

Above 

Range 


.Average 
10 

( % ) 


^Index" 


A 


1 n 


0 


0 


100% 


93 


3 


B 


7 


0 


0 


100% 


89 


3 


C 


11 


0 


0 


100% 


92 


3 


D 


4 


0. 


0 


• 100% 


92 


3 


E 


11 


0 


0 


100% 


95 


3 


F 


16 


0 




100% 


92 


3 


G 


7 


0 


- 0 ^ 


100% 


92 


3 


H 


5 


0 


0 


100% 


87 


3 


I 


flO) 








(88) 




Total 


71 


0 


0 


100% 


92 


3 




a 

Mathematics 




School 


No. of 
Classes 


% in 
■ Below 
Range 


% in 

Expected 
Range 


% in 

Above 

Range 


Average 
10 

(. t ) 


10 

^ , b 
Index° 


A 


10 


. 0 


30% 


70% 


93' 


3 


B 


6 


0 


33% 


67% . 


92 


3 


C 


11 


6 


45% 


55% 


93 


3 




7 


0 


43% 


57% 


83 


•3 


E 


8 


0 


13% 


88% ' 


91 


3 


F 


13 


0. 


' 31% 


69% 


92 


3 




3 


0 


0 


100% 


95 


3 


H 


4 


0 


25% 


75% 


• 90 


,3 


I 


. ai) 








(91) 




Total 


62 


0 


31% 


69% 


91 • 


3 



10 Index not appropriate for grades 7 and 8; averages for these grades 
not iricluded in totals 



Index Key 

1 = Below expected rate, ^' 

2 = At expected rate. ' 

3 = Above expected rate* 
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results are extremely high, particularly in reading where overlap for all 
classes in the district was in the range where student achievement would 
be expected to exceed the normal rate of growth. Classroom level results 
for mathematics, although very high, were not quite'^ as positive as those 
for reading. In summary, teachers indicated that coverage of contend in 
their classes matched almost all basic skills objectives assessed by the 
CAT. From this, one would expect growth in student achievement to rb.e very 
high. 

Academic performance . Attending to academic performance requires 
systematic monitoring of stiSdents' success in daily work, mastery of 
skills, and review of content. Although teachers were introduced to the 
concept of academic " performance in training sessions, this variable was 

V' 

not emphasized as strongly as SET or instructional overlap during the 
field test. Accurate data describing this variable were limited; teachers 
periodically described overall success and mastery rates for their Glasses 
on the principal/ teacher conference forms. Teachers at approximately half 
of the schools reported success. and mastery rates in the 80-100 percent 
range, with the remaining schools in the 50-79 percent range. Results 
were similar for reading and mathematics. In summary, the reported 
results indicate that students' success and mastery were relatively higb 
across the district, although the quality of the data is poor. No data 
were collected on students' review of content. 

Summary . In conclusion, the relatively high level of program 
implementation seemed" to , rls&lt in positive outcomes in terms of targeted 
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student behaviors and instructional processies. Based on the^e results, 
one would also expect students' achievement gains to exceed what mig^it be 
pi^dicted on the basis df national norms. 

Student Achievement 

Results from the district-wide testing program were used as the basis 
for examining student achievement in reading and mathematics. The . 
California Achievement Tests (CAT) were administered to all students. in 
every grade. Results are presented in Table 14 in terms of normal curve 
equivalents (NCEs) by school and grade, for. tests administered in the 
spring of 1980, 1981, and 1982. The grade level scores are reported in a 
diagonal analysis keyed to the groups of students in particular grades., in 
1982 so that groups are relatively comparable across years. For example, 
scores in the row for grade three Represent achievement of students in 
grade three in 1982, grade two in 1981, and in grade one in 1980. Thus,, 
scores can..be compared directly across" rows to assess overall achievement 
gains. However, since scores for grades one and two were not available 
for all three years, they were not included in any school mean. Table 14, 
also shows NCE gains over the baseline yr-r. (80-81) as well as over the 
field test year (81-82). 

According to the Title I ivorm-ref erenced evaluation nfodel, students' 
achievement should progress at a 'rate consistent with that of students in - 
the national norming sample.- In other words, an NCE change of zero indi-. 
cates that students' achievement growth rate equals the rate of the repre- 
sentative' national sample. The findings show tha't improvement in both 
reading and mathematics was demonstifated in the New Jersey ' School District 
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Tabiy 14 

Student'^ Achievement Scores^; New Jersey School District 



READING 



School A 



School B 



School C 



1982 

GRADE 

LEVEL 


1980 


1981 


0 

1982 


80-81 


%^ 
81-82 


1980 


1981 


1982 


+ ^ 
80-81 


81--82 


1980 


1981 


1982 


80-81 


81-82 


1 
































2 




54 


' 57 




, +3 




62 


61 




-1 




47 


50 




+3 


3 


48 


53 


50 


+5 


-3 


52 


57 


49 


+5 


-8 


44 


47 


51 


+3 


+4 


4 


46 


44 


46 


-2 ■ 


+2 


54- 


49 


56 




+7 


46 


51 


45 


+5 


-6 


5 


44 


45 


55 


+1 


+10 


57 


61 


56 




-5 


51 


43 


49 


-8 


+6 


6 


37 


43 


48 


-6 


+5 


53 " 


60 


57 


+7 


-3 


42 


46 


45 


+4 


-1 


7. 












56 


54 


66 


-i 


+12 












MEAN 


44 


46 


50 


■ +2 


+4 


54 


56 


57 


+2 


, +1 


46 


47 


48 


+1 


+1 




MATHEMATICS 



School A 



School B 



School C 



1982 

GRADE 

LE.VEL 


1980 


1981 


1982 


•^-81- 


+ ./ 

m-32 


i9«a 


1981 


1982 


'.80-81 


81-82 




1980 


1981 


-1982 


80-81 


+ 

81-82 


1 


































2 ■ 




57 


62 




+5 




57 


60 




-+3 






li3.. 


62 




+9 


3 ' 


55 


. 3 


, 60 


+3 • 


+2, 


56 


65 


58 


+9 


-7 




54 


62 


57 


+8 . 


■ -4 


4 


53 


52/ 


61 


-1 


+9 


65 


59 


67 


-6 


+8 




5'2' 


58 


.64. 


+6 


+e 


. 5 


58 


54 


'67 


-4 


+13 


69 


70 


66 


+1 


-'4 . 




51 


'53 


59 


+2 


+6'" 


6 


49 


57 


61 


^^+8 


+4 


64 


71 


77 


+5 


+6 




. 56 


56 


' 53 


a 


-3 


7- 










'. ^ 


67 


ff3 


77 


% -5 


+14 












■ > 


MEAN 


54 


1 — 2f 1 


62 


, -+1.(; 


+7 


64 


66 


6'r 


+2 


.-. -i -3 




53- 


57 


58 


+4 


+1 




Scores represent results of California Achievement Tests as notilial curve equivalen^is . 
(NCEs) compared through a diagonal analysis as described in text. School 'means were 
computed from grades wh^re data jwere available' for all three years. School means, 
therefctltre, include scores, from, grades three through six for Schools A, C, apd G, and 
scores from grades three through seven for s/hools B, D, E, F, and H, Complete data^ 
for 1980 and 1981 for the middle Qchool (School I) were unavailable because it 
received students from four elementary schools.and records were * dispersed and difficult 
to obtain. A mean for this school was not calculated due to the incompleteness of «the 
data. " ' V' * ' ,^ ' y 
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READING 



School D 



Table 14 (cont'd) 
New Jersey School District 
School E - 




School F 



1982 

GRADE 

LEVEL 


1980 


1981 


1982 


+ ^ 
"80-81 


+ ^ 
8l-8i2 


1980 


1981 


1982 


oO-ol 


ol-oz 


T no n 


1 QR 1 

OX 




OVJ OX 


o X o 


1 
























/■ - 








2 




51 


48 




■ -3 




53 


41 




-12 




62 


49 




-13 


3 


49 


53 


51 


+4 


, -2 


51 


53 


51 


+2 


-2 


54 


41 


40 


-13 


-1 


A 


47 


45 


51 


-2 


+6 


48 


47 


46 


" -1 

^ 


-1 


44 


38 


46 


-6 . 


+8 ' 


5 


42 


44 


48 


+2 


+4 ' 


47 


39 


40 


-8 


+1 


45 


' 41 


47 


-4 


+6 


6 


49 


47 


46 


-2 


-1 


45 


, 40 


45 


-5 


+5 


/44 


40 


38 


-4 


-2 


7 


52 


42 


49 


-10 


+7 


41 


40 


39' 


-1 


-1 


36 


39 


43 


+3 


.+4 


ME'AN 


48 


46 


49 


-2 


+3 


46 


44 


44 


-2 


0/ 


45 


40 


43 


-5 


+3' 






















/• 













MATHEMATICS 




School D 



School E 



School F 



1982 
GR4DE J 
LEVEL 


1980 


1981 


1982 


'80-81 


-81-82 


1980 


1981 


1982 


+^ 
'80-J81 


81^82 


1980 


1981 


1982 


"80-81 


+ 

81-82 


1 
































2 




53 


6Z 




+9 




59 


42 




—! 

-17 




64 


55 




-9 


3 


54 


62 


57 


+8 


-5 


55 


55 


57 


O- 


+2 


61 


48 


49 


-13 


+1 


4 


52 


58- 


64 


' +6 


+6 


50 


54 


53 


+4 


-1 


48 


46- 


54 


-2 


+8 


• 5 


51 


53 


59 


+2 


+6 


52 


44 


49 


-8 


+5 


So 


48 


58 


-8 


. +10 


6 


56 


56 


53 


0 


-3 


51 


45 


48 


-6 


+3 


54 


50 


55 


-4 


+5 


7 


58 


5^ 


60 


0 


+2 


47 


^55 


48 


+8 


-7 


48 


55 


56 


+7 


+1 


MEAN 


" '54 


57 


59 


+3 


. +2 


51 


51 


51 


0 


0 


53 


49 


54 


-4 


+5" 
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Table 14 (cont'd) 
New Jersey School District ' - 

School H ' / . School I 



LEVEL 


x980 


1981 


1982 


'SO- 81' 


81-82 


1980 


1981 


'19-82 


»^+^^' 
80-81 


+ / 
81-82. 


1980 


1981 


1982 


80-81 


/ 

81-82 


1 
































2 




55 


61 ' 




+6 




54 


57 




+3 












3 


49 


42 


64 


-7 


+22 


58 


50 ' 


51 


-8 


+1. 












A ■ 


43 


47 


58 


+4 


+11 


. 51 


■ 54 


50 


+3 


-4 












5 ' 


' 47 




. 44 


-6 


+3 


'52 


54 


59 


+2 


+5 












61 


4 ( 

48 


46 


53 


-2 


+7 


'51 


60 


:'58 


+9 


-2 












1 . 












52 


56 


56 


+4 


0 












& 
























'39 


42 




+3 


MEAN 


^7 


44 


55 


-3 


+11 


53 


55 


55 


+2' 


0 













■^^i|athematics 

i ■ School G . ' School H ^ School I ' 



J. — 1 — 
1982 

mil 


1980 


1981 


1982 


+^ 
"80-81 


+ /" 
'81-82 


1980 


1981 


1982 


+^ 
:80-81 


81-82 




1980 


1981- 


1982 


+. _■ 
'80-81 


81-82 








































56 


70 




+14 ' 




52 


55 




+3 














t 


'50 


45 


65 


-5 


+20 . 


56 


48 


59 


(1 

-8 


+11 
















45 


54 


'61 


+9 


+7 


"■ 51 


62 


59 


+11 


-3 














-J 


56 


47 


52 


-9 


+S 


57 


58 


59 


+1 


+1 
















50 


56 


65 


+6 1 


+9 


55 


6l| 


63 


+5 


42 


























52 ■ 


64 


67 


+12 


+3 




f ~ 




53 






meIj^ 




1 — 






















44 


46 




+2 




50 


51 


61, 


+1 


+10 


54 


' 59 


61 


+5 

















/ 




READINr. 

School G- 



/ 
/ 

/ 

/ 



during the field test year. In general, gains were slightly /higher in 
mathematics than in reading. At the school level, overall gains were 
•highest at schools A, F, and G. At these schools, total increases ranged 
•from- three to 11 NCE points in reading and from five to 10 NCE points in 

mathematics. No school registered a decline in achievement during the 

; 

field test jfear. Even in the "worst" cases, student achievement improved 
at a pace similar to that of the national norm group (NCE gain of zero) : 
Schools E and H in reading and School E in mathematics. In other words, , 
achievement a.t all schools progressed at a rate equal to or exceeding x^hat 
would have been expected relat^pa^o national norms. Educationally 
significant gains of five or md|e NCE points^ were made at one school in 
reading (School Ghand-at -three^sehoc^ls-in-ma^he^^^ 



G). , 

In reading, of the 46 grade lev^l score changes for the fi61d test 

year, there were 26 gains ranging from one to 22 NCE points; 19 losses, 
ranging from one to 13 NCE points^ and one case of no NCE gain. Fifteen 
of the 26 gains were educationally significant, while only five of the 19 
losses were educationally significant. In mathematics, there were 35 
Tgains, ranging from one to -20 NCE points; and 11 losses, rang-itig from one 
to 17 NCE points. Twenty-two of the 35 gains were educationally signifi- 
cant, while only five of the 11- losses were educationally significant. 
With the exception of School I, the middle school^ at least one grade 



^ ^See chapter on Ylelk Test Evaluation Procedures for discussion of 
rationale for rule of thumb considering educational significance. 
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level at each school made an educationally significant gain in reading 
and/or mathematics. 

From 1980 to 1981, of the eight schools for which data were 
available, in reading four schools demonstrated overall gains, from one to 

two NCE points, and four schools registered overall losses, from two to 

D 

five points. Three of the four schools that made gains in reading 
over 1980-81 also made" gains over 1981-82. The fourth school maintained 
its overall growth pattern (i.e., an NCE gain of zero). Three of the four 
schools that registered losses from 1980 to 1981 in, reading made gains ^ 
from 1981 to 1982; the fourth school maintained its overall pattern (i.e., 

a NCE gain of zero) . 

From 1980 to 1981 in mathematics six schools registered overall 
gains from one to five NCE points, one school registered an overall. loss 
of four NCE points, and one school had no NCE gain. All six schools that 
made overall gains over 1980-81 also made gains over 1981-82; the school- . 
with a loss from 1980 to 1981 made a substantial gain; and the school with 
no NCE gain between 1980 atid 1981 continued its overall pattern between 
1981 and 1982. Some of the changes were dramatic. For example, at School 
G, the school total score for reading changed from -3 NCE points over 
1980-81 to +11 NCE points over 1981-82, and the total mathematics score 
changed from +1 to +10 NCE -points. In addition, the mathematics score at 
•■^School F changed from -4 to +5 NCE points. 

Results on the state-wide basic skills proficiency test corroborate 
the gains and'high levels of achievement demonstrated on the CAT. As 
Table 15 indiCates/a higher peipentage of students ;in both elementary 
grade levels tested met statewide standards in 1982 than in 198P in 
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Table 15 



Student Achievement Relative to State Standards: " New Jersey School District 



H 



Percent of Students 
Meeting State Standards on Minimal Basic Skills Test 

Math 




3 
6 



.3 
6 



3 
6 



3 

6 



3 
6 



3 
6 



e 

3 
6 



71.7 
59.0 



73^.9 
85.7 



59.1 
78.8 



86.8 
74.2 



78.1 
85.2 



67.7 
80.3 



87.5 
92.3 



85.7 
100.0. 



74.1 
79.7 



1982 



86.5 
100.0 



76.5 
100.0 



100.0 
76.9 



97.3 
95.1 



92.3 
85.7 



92.3 
100.0 



100.0 
95.2 



84.6 

100»0 



92.9 
92.6 



Change 



14.8 
41.0 



2.6 
14.3 



40.9 
-1.9 



10.5 
20.9 



18.1 
0.5 



24.6 
19.7 



12.5 
2.9 



o -1.1 

0.0 



18.8 
12.4 



1981 



78.3 
68.4 



95.7 
82.1 



86.4 
79.2 



100.0- 
77.4 



75.0 
77.8 



73.8 
75.4 



90.6 
92.3 



92. -9 
100.0 



84.4 
79.4 



Reading 
1982 



94.6 
96.4 



94.1 
100. -0 



100.0 
88.5 



97.3 
85.4 



96.2 
91.4 



.92.3 
^81. 8 



Change 



16.3 
28.0 



-1.6 
17.9 



13.6 
9.3 



100.0 
95.2 



100.0 
93.3 



96.0 
89.6 



-2.7 
8.0 



21.1 
13.6 



18.5 
6.4 



9.4 
2.9 



7.1 
-6.7 



11.6 
10.2 
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virtually all schools. Improvements were evident in both reading and 
mathematics. At the end of the field test, almost all third and sixth 
grade students in the district met the standards. District level totals 
ranged fro?i 90 to 96 percent. These represented improvements ranging from 
10 to 19' percent, despite nearing the ceiling for the groups. 

In summary, student achievement results in both -reading and mathe- 
matics are impressive. Students at all' schools progressed at rates at 
least consistent with achievement expectations based on the national norm 
group, and at many schoola, gains exceeded achievement expectations. In 
most cases, these gains reversed trends exhibited during the baseline 
year. At the end of the 1981-82 year, achievement in most field test 
schools was around the national average in reading, and significantly 
higher than the national average in mathematics. Likewise, performance 
relative to statewide basic skills standards improved, with almost all 
students in the district meeting state standards appropriate for their 
grade level. 

Relationship of Classroom Process Variables to Student Achievement 

Achievement Directed Leadership is based on reseajch that suggests 
strong relationships between several classroom process variables and 
student achievement gains. Therefore, it was expected that student 
achievement would be directly related to the degree to which teachers 
actually attended to these variables in their classrooms. To test the 
relationship between classroom process variables and student achievement 
gains, indices were constructed as outlined in the field test methodology 
chapter, and chi-square analyses of available SET, instructional overlap , 



and student achievement data, from the New Jersey School District were per- 
formed. Due to the lack of variability in the classrooi.. process data 
(i.e.', almost all scores were ia the extreme high end of the range), no 
relationship was^'.evident . However, this does not refute the research 
base, since variability must exist for correlations to be established. 

Rel ationship of Level of Program Implementation to Student Achievement 

'This subsection examines the relationship between level of program- 
implementation and student achievement gains in the New Jersey School 
District. Table 11 on page 73 summarized the level of program implemen- 
tation for each' of the field test schools. Table 16 shows how these 
implemention summary indices are related to student achievement gains. 
-The results are graphically illustrated in Figures 5 and 6. for reading and 
mathematics, respectively. Since no school mean for student achievement 
.was calculated for School I, data far that school are not included ia 
Table 16, or in Figures 5 or 6. The findings demonstrate relational 
trends ^between level of program implementation and student achievement . 
The two schools (Schools A and G) with a high degree of program 
implementation exhibited the largest increases ia a^^hievement in i^eading 
and mathematics ovfer the course of the field test. In addition, tlieir 
gains from 1981 to 1982 were in contrast to their gains over the baseline 
year (1980 to 1981). On the other hand, the three schools (Schools B, C, 
and H) with a low degree of implementation exhibited little change during 
the field tes'f year and this change was basically consistent with the 
change over the . ^ 




.Table 16 

Sununary' of Relationship Between Level of Program Implementatl6n 
gnd Student Achievement Gains: > New Jersey School District 



Note: Test 'used Is the California Achl^evement Test: Scores are reported as NCEs . 




School 


Summary of 
School Imple- 
mentation 


1980 


1981 


1982 


8{)-81 


81-82 


1980 


Ivol 


1982 


+/- 
80-81 


01 Q O 


A 


High 




A6 


50,.^,.^ 


+2 


+A 


5A 


55 


62 


+1 


1*7 
+ / 


B 


Low 


5A 


' 56 


57 




+1 


6A 


66 


69 


+2 


+ J 


C 


Low 


S 

M D 


A7 


A8 / 


+1 


+1 


■ 53 


57 


58 ' 


. +^ 


+ 1 


D 


^ Medium 


A8 


A6 


A9 


-2 , 


+3 


5A 


57 • 


59 


+3 


+2 


E 


Medium- 


A6 


^1 AA 


AA 


-2 


0 


51 


51 


51 


0 ' 


0 


F" 


Medium 




AO 


A3 


-5 


+3 


53 


A9 


5A 


-A 


+5 • 




High 


A7 


4A 


•'55 


-3 • 


+ 11 


5Q . , 


•51 


61 


+ 1 


+ 10 


' H 


Low 




55 


55 


+2 


0 


5A 


59 


61 


+5 


+2 


MEAN ' 




AS 


A7 


50 


-1 . 


+2 


5A 

1 


56 


59 


+2 \ 


+A 



ERIC 
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READING 



70 



60 



AO 



High 

Implemen t a t ion 



School G 




School A 



Medium 

Imp 1 emen t a t i on 



School D 




School F 



Low 

Imp lemen t a t-ipn 



School B 




School H 



School C 



1980^ 1981 1982 , 1980 1981 1982 1980 1981 1982 



Figure 5., Relationship of implementation to achievement gains in reading 
by school: New Jersey . School District. 
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MATHEMATICS 



I 



High 

Implementation 


Medioom 

Implementation 


Low 

J Implementation 

School B 








■School A 










School H 




School D 


• ^r'^^chool C 


7 / / School 




^^^^^ 


^^^^^^ / 


Schpol Fy 
^ School E 




4 






■t 

t 

% 

L r ^ i L_ 


► 




1980 19B1 1982 


1980 1981 "1982 


1980 1981 1982 ' 



Figure 6. Relationship of implementation to achievement gains in- mathematics 
by school: New Jersey School District. 
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baseline year. ,^ The three schools (Schools- D, E, and F) with medium levels 
of implementation exhibited varying achievement gains during the field 
test year and in comparison to the baseline year. In some cases, their 
gains (e.g., School F for mathematics) approached those of the high 
implementation schools while in other cases their gains (e.g.. School E 
for reading and mathematics) were similar to low implementation schools. 

In summary, a relationship between l^el of program implementation 
and student achievement in reading and mathematics was demonstrated. 
Although all' schools implemented th^ pragraro to some degree and all 
exhibited improvement at least consistent with expectations , achievement 
gains were most positive for those schools with the highest levels 'of 
implementation. Although the field test evaluation design was 
non-experimental, this relationship lends considerable support to the 
hypothesis that Achievement Directed Leadership has a considerable 
positive '.mpact upon student achievement. 

Overall Summary of New Jersey Field Test Findings ; 

This chapter has described Achievement- Directed Leadership field t^s 
findings in the New Jersey School District in terms of implementation 
activities and outcomes at .five levels: BSC, district, principal, 
teacher,, and student. The district ' s eight elementary schools and^.one- 
" middle school participated in the field test. TheNew Jersey School 
District was selected as the site for the field test because of its plans 
for comprehensive implementation. 

Orientation and initial training of central office staff and 
principals were regarded as successful. Training was well received and 
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adminstrators were confident thato trhey had acquired the necessary knowl- 
edge and skill's to implement all components of the program. However, BSC 
observers reported that levels of ^principal commitment and . understanding - 
of ke-y concepts varied somewhat, implying that teacher training and 
progr§im implementation would vary in quality among schools. 

The district superintendent and assistant superintendent provided key 

leadership in setting district-wide policies for school improvement^ arid 

' . ^ * • 

too]^ an active role, in planning and implementing the program. Other 

central office staff were also actively involved as they assumed jiew roles 

as instructional leaders. All central office staff were Very committed to 

the improvement approach and provided the support necessary for its 

success. All district level planning and participatory supervision , 

activities were effectively carried out. Although' the BSC initially 

provided considerable assistance in planning and training, the super- 

intendent, assistjant superintendent, and other central\of f ice staff 

gradually assumed major responsi|Dilities for these roles. 

Program implementation at the principal level generally proceeded 
acco'rding to district plans although some variation among schools was 
observed. Principals systematically planned for implementation, in most 
cases trained • teacher^s with respect /Ho the time and .content variables, and 
participated with teachers in supervisory activities. However, actual 
degree of implementation ranged from low (at three schools) to high (at » 
one school). -These variations are probably due, in large part, to the . 
differing levels of commitment and understanding noted by BSC observers. 

Overall, teachers' activities sufficiently followed the designated 
implementation model when considered on a district-wide basis/ For" ~ . 
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example, they used research fD^guide instructiot\al improvement, and 

attended ^to targeted classroom- variables. They also, reporTeB that they 

' ' .. 

were" generally successful in implementing the ^pragram cotaponents in th'eir 

i , - - 

classrooms. However,' as with princiijpars', levels of. commitment and imple- 

- _ . - , 

mentation seemed to vary widely across schools • Actual degree of teacher- 
implementation ranged from low (at three schools) to high (at two 
scHools/) . Considering the joint Effects of principal and teacher t> 
implementation," Schools were categorized as relatively high (2 schools),, 
medium (4 schools), or low (3 schools) in teras of overall levels of 
implementation of the itnprovemept approach. . . r 

The moderate level of program impleiuentation district-wide S3emed to^ 
result in positive outcomes in terms of targeted' student behaviors* 
Instructional overlap between actual classroom instruction and leariiing , 
objectives corresponding to standardized achievement test it;ems was . 
uniformly high throughout schools in the district. L|.kewi^e, teachers at 
most schools met their targeted goals for maintaining high levels of ^ 
student engaged time. Although measures of other important classroom, 
process variables were not 'availably, conditions '.were exhibited that would 
indicate, high expectations regarding student achievement gains. <> 

Student a'dhievement results in both reading and mathematids were" 
impressive. Students at all schools progressed at rates at least con- 
sistent with achievement expectations based on the national nortn group, 
and at many schools j gains exceeded achievement. expectations . In most 
cases these gains reversed trends , exhibited during the baseline year. 
District-wide achievement,^ at the end of the field test, was around the 
national average in' reading and significantly exceeded the national 
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average'^ in matKemafeicSe Likewise,^ most students in the district met 
statewide minimum basic skills"'^s4:and,ards 

^ 'A relationship between leveT of program implementation and student 
achievement in reading 'and '^fnidthematics was observed, Achievemejit *gains 
w^re most positive for those schools with the higliest levels of Itcplemen- 
.tation and least positive (although i^pt negative) for schools with lowest 
levels of implementation. These relationships lend considerable support 
to the 'hypothesis that Achievement Directed Leadership h^s a significant 
positive impact or\ student achievement, . 
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PENNSYLVANIA ANALYSIS 



The 1981-82 field test of the effectiveness of the Achievement 
Directed Leadership program was conducted by the Basic Skills Component 
(BSC) of Research for Better Schools, Inc., in three school districts 
located in New Jersey, Pennsylvania, and Delaware. Tie New Jersey 
District was the focus of the field test since the program was fully imple- 
mented in this district during the time of the field tes.t. 

The field' test in Pennsylvania and Delaware consisted of supplemen- 
tary analyses to examine additional questions about the effectiveness of 
Achievement Directed "Leadership . The focus of interest in the 
Pennsylvania District was the effectiveness of the program when only part 
of the total approach is implemented in a school district. The major 
interest in the Delaware District was the continuing effects of program 
participation on student achievement over a number of years. 

This discussion of the Pennsylvania analysis is in five sections. 
First, the purpose of the analysis is outlined. Second, an overview of 
the school district and the participating schools is presented. Third, 
the nature and extent of program implementation are described. Fourth, 
student achievement results are displayed and discussed. Fifth, con- 
clusions concerning partial implementation are presented. 

Purpose of the Analysis 

" During the summer of 1981, the BSC and the Pennsylvania District 
discussed the possibility of the district's involvement in the 1981-82 
field test of Achievement Directed Leadership. The district, which had no 
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prior involvement with the program, expressed interest in a partial imple- 
mentation. The BSC viewed the district's interest as an opportunity to 
test whether partial implementation of Achievement Directed Leadership 
" would influence student achievement, and decided to include the district 
in the field test for that reason. Thus, analysis of the Pennsylvania 
District's involvement focused on determining whether partial implementa- 
tion of the program would affect student achievement. 

District and School Overview 

The Pennsylvania District has 12,700 students from an industrial city 
and its surrounding boroughs. It is the fifth largest school district in 
the state. It has 17 elementary schools, five junior high schools, and 
two senior high schools. The district's minority population, about 12 
percent of the total student population, is predominantly Hispanic. 

.Table I'presents some descriptive characteristics of the five elemen- 
tary schools that participated in the field test. All of them are K-6 
schools, and there is little variation among them in the other character-^ 
istics listed, although School E is a smaller school with a lower average 
daily enrollment and Schools B and D have a higher average class si^e. 

Nature and Extent of Program Implementation 

In August of 1981, the BSC conducted an orientation of Achievement • 
Directed Leadership for some central office staff and elementary 
principals in the Pennsylvania District. After the orientation, it was 
decided that the district ' s- implementation would consist of the following: 
(1) the BSC would train principals of five of the 17 elementary schools 
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Table 1 



Description of Field Test Schools; Pennsylvania District 





\ 

Schools 


Charact eris txcs 


A 


B 


C „ 


D 


E 


Grades in School 


K - 6 


K - 6 


K - 6 


K - 6 


K - 6 


Average Daily Enrollment 


295 • 


262 


331 \ 


286 


165 


Average Daily Attendance 
Rate 


95% 


91% 


92% 


90% ■ 


93% 


Number of Teachers 


10 


8 


13 


9 


6 . 


Average Class Size 


26 


31 


25 


32 


25 


Average Years Experience 
of Teachers 


over 10 


over 10 


over 10 


over 10 


10 
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and .some 'central office staff in the 'improvement approach and in the 
management of the time and content variables; (2) these principals, with 
BSC and central office support and assistance, would train and work vlth 
teachers in the use of the improvement approach and the management of th^ 
time and content variables; and (3) the BSC would work with central office 
staff in leading and supporting the work in the schools. 

In August,, the .BSC conducted a five-hour introductory training 
session on content management. Two of the five principals and some 
central office staff attended this session. Because the district lacked 
prerequisites -for implementation of content management (i.e., prior 
learning data^ and a standard curriculum or common texts for both reading/ 
language arts and mathematics) , teacher training in content management was 
to be limited in scope and postponed until necessary materials became 
av^ailable. The BSC provided further training to principals in content 
management during monthly leadership seminars. 

In September, the BSC conducted a time management workshop. Twelve 
hours, spanning three days, were devoted to this workshop. The five 
principals and central office staff assuming major roles in program 
implementation attended all three days. 

After the time training workshop, the five principals oriented and 
trained teachers in their respective schools in the improvement approach 
and in the management of the time variables. Most of the formal time 
training of teachers by principals was conducted during faculty meetings. 
This time training was completed by November. 

In regard to content management training, as needed material became' 
available, it was provided to principals who in turn informed teachers and 
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monitored their use of these materials. For mathematics, the content 
forms matching te^st items with curriculum were completed by the BSC by 
December, However, similar forms for reading were not completed until 
April. The principals in Schools A and E conducted the earliest and most 
intense work with teachers in ,the management of the content variables for 
mathematics. The other principals did varying amounts of informal work 
with teachers on an individual basis in regard to mathematics content 
training. Limited training was provided to teachers in the content 
variables for reading during the time of the field test. 

After they Y ad trained,^ their teachers in the time variables, the five 
principals monitored teacher performance of program functions and engaged - 
in ongoing communication v7ith teachers primarily through structured class- 
room observations' and principal/teacher conferences. Most teachers 
reported that principals observed their classes approximately seven times 
and conducted an average of three conferences. Both principals and 
teachers reported that teachers used the improvement approach in their 
classrooms. 

In addition to training, the BSC provided ongoing support and 
technical assistance to central office staff And the five principals. At 
least once a month, the BSC conducted leadership seminars for the involved 
central office staff and the five principals. These seminars do.alt with 
training needs, implementation problems, and planning issues. The BSC 
also provided follow-up technical assistance through periodic meetings 
with central office staff and with individual principals who requested 
assistance. 

r 

P 



The sup'^rintendent of the Pennsylvania District conducted one formal, 
conference With each of "the five principals involved in the program. 
These conferences provided an opportunity for the superintendent and 
principals to discuss any program concerns and to review plans and activi- 
ties for instructional improvement. 

In.sjimmary, partial implementation of Achievement Directed Leadership 
in the Pe/nnsylvania District basically entailed the following: 

1. the BSC oriented and trained five principals arid some central 
office staff in the improvement approach and the management 
/of the time, and content variables. 

2. /The five principals, with BSC support and some assistance by 
I the central office' staff, oriented, formally trained, and 

; worked with their respective teachers in regard to the 

\ improvement approach and the management of the time 

\ variables. ' 

.3'. All of the principals formally or informally trained and 
/ worked with their respective teachers , although to varying 
' degrees, in regard to the management of the content variables 
for mathematics. Work done by principals in the management 
of the content variables for reading was limited in scope and 
accomplished late in the school year. 

Student Achievement Results 

• Results from the district's testing program Wbre used as the basis 
fot examining student achievement in reading and mathematics. In the 
Petinsylvania District, the Science Resarch Associates (SRA) achievement 
t^st series is administered to all students' in every grade each spring. 
^RA test results are summarized in Table 2 in terms of normal curve 
equivalents (NCEs) by school and grade, for tests "admintstWe^ 
and The grade level scores are reported in a diagonal analysis 

keyed to the groups of students in specified grades in 1982 so that groupi 
are relatively cotn^arable across years. For example, a score in the row 
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^ Table 2 

Student Achievement Scores : Pennsylvania District 



Reading 






55rhool A 


School B 


School C 


School D 


School E 


Grade 

Q 1 QD 


1981'' 


1982 ^1-82 


1981 


1982 


^+/^ 
81-82 


a98i 


1982 


81-82 


1981 


1982 


81-8Z 


1981 


3982 


81-82 


1 




76 






6A 






A8 






59 






67 




2 


72 


80 


+8 


6A 




-5 






+8 


67 


62 


-5 




50 


+3. 


3 






0 


51 


52 


' +1 


62 


58 


-A 


60. 


60 


. 0 




63 


-1 




58 


63 


+5 


59 r. 


58 


-1 


53 


57 


+A 


55 


. 5A 


-1 


49 


66 


-5 


5 


59 


61 


+2 


A6 


53 


+7 


A8 


51 


+A 


62 


66 


+A 


3A 


69 


+15 


6 


57 


66 


+9 


58 


55 


-3 


•61 


57 


-6 


68 


66 


-2 


kl 


69 


+7 


MRAN 


60 


6A 






^ 55 




56 


56 


0 


61 


62 


+1 


kl 


66 


+6" 




w ' — . 


■ • Mathematics 






School A 


School B. 


School C 


School D 


School 


E 


Grade 
81-82 


1981 


1982 


+ 

•> 

81-82 


1981 


1982 


+ . 
81-82 


1981 


1982 


>- 

81-82 


1981 


I982f 


+ . 
81-82 


1981 


1982 


+ 

81-82 . 


n 




79 






75 






^8 






65^" 






66 




2 


63 


81 


+18 


72 


66 


-6 




68 


+20 


68 


60 


-8 




52 


+6 


3 


63 


69 


.+6 ■ 


60 ^ 


59 


-1 


55 


57 


-3 


66 


69 


+3 


. AO 


61 


+1 




70 


75 


+5 


59 


56 


-3 


^9 


5^ 


+5 


60 


66 


+6 


Al 


56 


+15 


5 


66 


71 


+5 


A7 


59 


+12 


^2 


50 


+8 


70 


75 


+5 


65 ■ 


50 


■ +5 


6 


72 


•82 


+10 


50 


59 


+9 


57 


58 


+1 


73 


73 


0 


69 


55 


+6 


MEAN 


ft8 


7A 


+6 , 


5^ 


58 


+^ 


51 


5A 


+3 ■ 


67 


71 


+A 




51 


+7 



Scores represent results of Science Research Associates (SRA) Achievement tests as normal curve equivalents 
(NCEs) compared through a diagonal analysis as described in te^t. Total scores were computed from grades 
where data were available for all three years. School means, therefore, include scores from grades two 
through six for all schools. 



for grade three represents achievement of students in'grado three in 1982 

arid in grade two in 1981. Thus, scores can be compared directly across 

rows to assess overall achievement gains. These test scores are 

graphically described in Figure 1. " , 

A basic assumption of the Title I Norm-referenced Model is that 

student achievement progresses at a rate consistent witji that of students 

• < 

in the national norm group if no special treatment is introduced. In 
Other words, an NCE gai'TToTf z:ero ii^dicates that students' gain in - 
achievement equals that of the representative national ^ample. ^Although 
many "rules of thumb" have been used to determine the educational 
significance of achievement test score gains, £or the purpose of this 
report a change of five-NCEs was determined to be educationally 
. significant based on a statistical examination of the .differences between^. 

grade level means. ^ 

The following discussion refers to gains/losses during 1981-82, the 
field test year, for five grade levels, grades two through six. Grade one 
is .not included in the analysis because its 1981 achievement scores are 
not available. 

At the school level, the results in reading show that four schools 
registered gains ranging from one to four NCE points and one school 
registered a zero NCE gain. In mathematics, all five, schools registered 
gains ranging from three to seven NCE points. In Schools A and E, the 

mathematics gains reached the level of educational significance (i«e^, 

five or more NCE points). ' ^ . " 

Considering each of • the five grade levels \in each of the five 
schools, in reading there were 13 gains, ranging from one to 15 NCE 
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Schopl A 



School B 



School C 



Scbool D 



School E 



Figure 1- SRA reading (R) and^math (M) scores as NCEs for 1981 and 1982, for 
the five field test schools in the Pennsylvania District. 
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points; ten losses, ranging from one to five NCE points; and two cases of 
zero NCE gains. In mathematics there were 19 increases, ranging from one 
to 20 NCE points ;^ive decreases, ranging fr.om one to eight NCE points;- 
and one ca(se)of zero NCE gain. In terms of the educational significance 
of these grade level changes, in reading there were .seven cases of 
educationally significant gains and 'three cases of educationally signifi- 
pant losses. In mathematics, there werc^ 16 cas,es of educationally 
significant gains and two cases of educationally significant losses. 

In summary, during the field test year there wer^ increases in ^ 
reading and mathematics achievement^ at both overall school and individual 
grade levels. These achievement increases xi^ere greater and more frequent 
in mathematics than in'reading. Schoolwide, the two educationally 
significant gains were for mathematics,* at Schools A and E. When 
considering all 25 grade levels (i.e., grades two through. six) sixteen 
demonstrated educationally signficant gains for mathematics and seven 
demonstrated educationally significant gains for reading. 

Conclusions - ' v 

s 

^In the partial implementation of Achievement Directed Leadership by 
the Pennsylvania District, five elementary school principals were" trained 
by the BSC in the improvement approach and in the management of the time 
and content variables. After time training, these five principals 
oriented, trained, and worked with their respective teachers in regard to 
the improvement approach and the time variables. After being provided 
with prerequisite materials, these principals conducted some training and 
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^/orked with teachers in regard to the managem^t of the content •^'^ariables 
for' mathematics, and to a much lesser extent for reading. 

Student 'achievement findings from, the five participating schools 
suggest that the partial implementation of Achievement Directed Leadership 
in this district did positively influence student achievement in reading 
and mathematics. Moreover, student achievement gains for mathematics, an 
area that received greater program attention, were higher than those for 
reading. ' ' ' 
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Appendix B: 
Delaware Analysis 
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DELAWARE ANALYSIS 

The 1981-82 field test of the^ effectiveness of the Achievement 
Directed Leadership program was conducted by Research for Better Schools, 
Inc., in three school districts located in New Jersey, Pennsylvania, and 
Delaware. The New Jersey School District was the focus of the field test ^ 
since Achievement Directed Leadership was fully implemented in the 
district's nine elementary schools and one middle school during the time 
period scheduled for the field test. 

Two supplementary analyses were conducted in Pennsylvania and Dela- 
ware to examine additional questions about the improvement program. The 
focus of interest in the Pennsylvania analysis was the effects of Achieve- 
ment Directed Leadership'- when only part of the program is implemented in a 
school district. The major interest in the Delaware analysis was the 
effect of multi-year program participation on student achievement. 

This appendix discusses the Delaware analysis in five sections. 
First, the purpose of the analysis is outlined. Second, an overview of 
the district and field test "schools is given. Third, the nature and 
extent of implementation^^ of the program in the district are described. 
Fout^th, student achieveme'nt results for this school district during the 
period of program implementation are shown. Fifth, the possible effects 
cf continued program participation on student achievement are discussed. 

% . 

Purpose of the Analysis 

A field test of Achievement Directed Leadership' was scheduled for the 
1981-82 school year. Three school districts in different states agreed to 
participate iti the field test. The extent of prior participation in the 
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program and the level of planned involvement during the field test year 
were important factors that influenced the nature of the field test in 
each district. 

In Delaware, the cooperating school district had participated in the 
program for three years, and planned to maintain a similar ^level of 
involvement during the field test year. Thus, student achievement data 
would be available for four years of participation in the program in this 
district. These data would provide an opportunity to examine the effects 
of program participation on student achievement over a pexiod of time. 

The field test in this school district was designed to: (1) collect 
data on program implementation and student achievement during the field 
test year; (2) combine these data with data collected on program implemen- 
tation and student achievement in the previous three years; and (3) 
analyze the changes in student achievement over time in relationship to 
program implementation. . 

District and School Overview 

The cooperating Delaware site is a small rural district serving a 
population with low socioeconomic status. It has 2,136 students in one 
kindergarten school, two elementary schools, one.middle school, and one 
high school. Minority group students make up about 20 percent of .the 
student population. Table 1 provides an overview of the major character- 
istics of the two elementary schools (Schools A and B) which participated 
in the program. As the table indicates, both schools contain grades 1-5, 
but School A is a much larger school than School B and has more teachers. 
The teachers in School B, however, ,>ave more teaching experience. 
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Table 1 

.Description of Field Test Schools: Delaware School District 





Characteristics 


School A 


School B 




Grades in School 


1-5 


1-5 




Average Daily Enrollment 


510 . 


280 




Average Daily Attendance Rate 


90% 


94% 




Number of Classroom Teachers 


27 


13 


• 


Average Class Size 

"1 =i 


25 


25 




Average Years Experience of Teachers 


5 


11 
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Nature and Extent of Program Implementation 



As 



mentioned earlier, the Delaware School District had a three-^year 



history of cooperative development of the Improvement approach prior to 
the field test In the 1981-82 school year. This cooperative development 
Involved principals and teachers from all five school's, but participation 
was greatest at the tv/o elementary schools, especially School B. Table 2 
summarizes the Involvement of the central office and these two elementary 
.schools over a four-year time period during the development and^ field test 
of the approach. Some training was delivered every year In order to 
accommodate new participants, reinforce previous training for experienced 
program participants, and train all who would be Involved in new portions 
of the program as they were developed. In addition, the superintendent 
and his staff gave continued support to thei concepts of the program 
throughout this period and they facilitated related actlvltlea when 
opportunities arose. 

During the second half "of the first year of Involvement , administra- 
tors and some teachers at Schools A and B Implemented the time management 
portion of the Improvement approach. In January of 1979, some teachers at 



both elementary schools were oriented to the approach and trained in the ■ 
managementof students' use of classroom time. Observations were subse- 
quently carried out by administrators at both schools. At both schools, 
teachers met with their observers and RBS consultants to discuss the 
observation process " and subsequent use of data for imprpvement purposes.^ 
In these meetings, they compared the observation data to research find- 
ings, identified opportunities for improvement, selected improvement 
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Table 2 

Implementation of Achievement Directed Leadership 
in the Delaware School District 
1978 - 82 



Level 



Central Office 



School 
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1978 - 79 

Orientation 

Planning 

Time training 
and 

implementation 



Orientation 

Planning 

Time training 
and 

implementation 
(Schools A and B) 



1979 - 80 



Planning 



Orientation 

Planning 

Time training 
and 

implementation 
(School B) 



1980 - 81 



Planning 

Content 
training and 
implementation 



-4 no 



Planning 
Time 

implementation 
(School B) 



Content training 
and 

implementation 
(reading, Schools 
A and B) 



1981 - 82 



Planning 

Time and content 
training and 
implementation 



Planning 

Time and content 
training and 
implementation 



(reading and 
mathematics, 
Schools A and B) 



strategies, and planned for implementation of these strategies. Teachers 
then implemented the strategies in their classrooms. These meetings also 
provided useful suggestions tp RBS for developmental revisions of the 
program. 

During the summer of 1979 it was decided that RBS wduld work 
primarily with the staff of School B in time management during the 1979-80 
school year. In the fall of 1979, the principal of. School B and RBS staff 
reviewed the approach with all basic skills teachers and^ then trained 
seven of these teachers as observers for time management. These teachers 
observed each other's classrooms early in the fall, and met with their 
principal and each other in planning conferences. Teachers then imple- 
mented improvement strategies. The principal observed these classrooms 
later in the school year to assess the effectiveness of the improvement 
strategies and held a second round of conferences with teachers for review 
and planning purposes. * 

Plans for 1980-81 included continued implementation of the time man- 
agement component of the program at School B, and implementation of the 
content management materials in reading at both School A and School B. 
Consequently, reading teachers at both schools were trained by central 
office staff in the management of content variables, especially prior 
learning. While central office staff and principals monitored students' 
coverage of the curriculum throughout the school year, teachers at both 
schools attempted to remediate student prior learning deficiencies. In 
addition, two rounds of time management observations, and two rounds of 
conferences were held at School B. 
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School year 1981-82 was the year of the field test. Both schools 
agreed to utilize the time and content management materials in reading and 
mathematics. Accordingly, central office staff and administrators from 
the 'two schools received review training in the overall approach and in 
time and content management. These sessions were led by BSC staff during 
the summer and early fall of 1981. Teachers participated in the content 
'training session, but feviewed time management at a later date with the 
principals and BSC staff. Subsequent implementation of Achievement 
Directed Leadership was relatively high at School B, but only moderate at 
School A. Approximately two rounds of observations, and two rounds of 
conferences were held at School B; one round each of observatioris and 
conferences was held at School A. 

Student Achievement Test Results ^ 

^ In 1978, students in the Delaware School District were scoring around 
the national average, but well below the state average on the statewide 
standardized achievement tests. As noted above, RBS began to work in this 
district in January of 1979. Test results in the spring of 1979 showed 
high initial gains. Although several grades in the district registered 
decreases in 1980, these gains were, for the most part, reestablished in 
1981 and sustained or extended in 1982. ' ' 

Table 3 presents the average scores, for grades 1-5 on the total basic 
skills battery of the California Achievement Tests (CAT) across five years 
(1978-82) in the Delaware District. The table also shows the change in 
the average scores registered from one testing year to the next. Scores 
are reported as normal curve equivalents, or NCEs. Except for the 1978-79 
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Table 3 , 

Sttt'dent Achievement Scores: Delaware School District 



Grade Level 


Total Scores^- 


Score Changes 


Fall 
1978 


Spring 
1979 


Spring 
1980 > 


Spring 
1981 


Spring 
1982 


78-79 


79-80 


80-81 


i 

81-82 


1 


'49 


71 


61 


65 


* 

65 


+22- 


-10 


+4 


0 


2 


52 


64 


67 


65 


65 


+12 


+3 


-2 


0 


- 3 


52 


62 


58 


63 


63 


+10 


-4 


+5 


0 


4 


52 


61 


• 63 


64 


65 


+9 


+2 


+1 


+1 


5 


56 


65 


60 


62 


69 


+9 


-5 


+2 


+7 



Scores on total battery of California Achievement Tests (Basic .Skills) in normal curve equivalents 
for Schools A and B in Delaware School District, Except for the fall of 1978 and. the spring of .1979, 
data represent the performance of different groups of students in each succeis^sive year. 
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^chool year when test scores were available for the same students in fall 
and spring, the scores in the horizontal rowsufor grades represent differ- 
ent groups of students in each successive year. 

A basic assumption . of the Title I norm-referenced model is that 
student achievement progresses at a rate consistent with that of students 
in the national norm group if -no special treatment is introduced. In 
other words, an NCE gain of zero indicates that students' gain in 
achievement equals that of the representative national sample. Although 
many "riiles of thumb" have been used to determine the educational signifi- 
cance of achievement test score gains, for the purpose of this report a 
change of five NCEs was determined to be educationally significant based 
on a statistical examination of the differences between grade means. 

As Table 3 indicates, students in all five grades performed much 
better on the CAT in 1979 than they had in 1978. The gains, each of which 
was educationally significant, ranged from nine to -22 NCE points- In the 
following year (1979-80), three grades (grades one, three, and five) 
registered decreases, two of which were educationally significant. How- 
ever, in 1980-81 all three of these grades registered gains, g;ne of which 
was educationally significant, and in 1981-82 one of them registered an 
educationally significant gain while the other two had no NCE gains. The 
other two grades (grades 2 and 4) did not register any changes large 
enough to be considered educationally significant over the three years 
following the 1979 testing. These test scores are displayed in graphic 
form in Figure 1 . 
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Student Achievement Scores 
Delaware School District 
1978 - 1982 



o 



80 



75 



70 



65 



60 



55.* 



50 



65 



60 



grade 



grades 

2,3,6, 



grade l. 




. 1978 



1979 



1980 



1981 



1982 



Figure 1: California Achievement Test results as NCEs; grades 1-5 
across five years. Data reflect students^ performance 
on the total *battery of basic skills test's. 
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DlsccL.aslon of^ Score Changes 

There are several difficulties associated^ with relating achievement 
test score changes in the Delaware District to participation in the , 
Achievement Directed Leadership program. First, the nature and extent of 
implementation of Achievement^ Directed Leadership in these schools changed 
from one year to the next. Second, while , implementation at times was 
limited to one of the two schools, and at times to one subject area 

' (reading) , the achievement test score data presented here refer to the 
performance of students from both schools Oti the total battery of basic ^ 
skills tests. Third, the improvement approach was not the sole 
educational intervention in the district. All of these factors could b*e - 
used, to refute the argument that gains in student achievement test scores 
can be attributed to the district's involvement in the program. 

However, student achievement test, scores in the Delaware District 
rose markedly during the first year of participation in the program, and 

-most of the gains registered in 1979 seem to have been sustained over the 
years of .further development and field testing of approach. Again, 

although no direct correlation between student achievement test score 
gains and program implementation is established by thege data, it is* 
likely that Achievement Directed Leadership was responsible, in part, for 
the improvements in student achievement test scores. 

Indeed, the district's superintendent corroborated this assertion in 
November of 1981, stating that the "improvements [in achievement test 



scores] have been satisfying for all concerned and are no doubt attribut 
able to the hard work of teachers and students, to the opportunities pro 
vided ^^eachers for upgrading their professional skills, and to the 
assistance given to ^the district by RBS. " 
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Appendix C : 
Data Collection Forms /Instruments 




ERIC 



137 



ill 



CONTACT REPORT MEMO 

TO: D. Helms/A. Graeber 

FROM: * mJXoJi'i, Wome 

CONTACT: State " Sit e (Re/neinbM. no school YimQ^--iKi2. codu ) 

"Date of Contact h\onth, Vote.. ^eoA 

Meeting □ -Phone Call pl Tim e '{Mot jpfi phon e, cam] 

Participants: Mho paJuUcipatzd MWrXXXZoA' ot~n2M) an 
fioAzLy appzafilnQ panXlcipantii . 



DATE: 



Month, Vcutz, Vzoa 



la. 



lb. 



^n objccXlvz oi thz mzojtinQ I contact:, IncladlnQ mponXant pfLzcuAAou 
and pfizpan.aU.onA>. Oi zxXAmz mpohtancz aAn youJi 6tAatzg.Lu and 
njatlonaZz ion. achieving the. objective.. 

He.xt objective., U any, including impontant pn.e.cahA>on^ and pn.e.pan.a- 
tloYU,. The 6tAate.gieA and n.aXi.onale. planned ipt achieving the. 
obj-ective. 



2a. 

2b- 
etc. 
3. 



{iJhat happened with n.eJi.pect to la., tnjan^a(iXJ.on& , and oatcomei,. 
What happened mjth n.e6pect.to lb., tnjxn^ actio n^ , and outcomes. 



Othen. Inionmatioh learned duAlng the call that li> not diAecXly 
related to an objective ci the contact. Thei,e can be dUcuA^ed ^n 
paA.agn.aph 6tyle and do not need to be labeled with letteu, Id no 
inionmatton woa collected, type 3. N/A. 

4. What I {the mlteA) need to do 06 a neMxlt o^ thU. contact. iHhat . 

li> my problem- 4>olving 4>tAategy? {e.g., A&k V. Helin& to call M^-. Adorns; 
Vnlnt 10 copies oi the Onlentatlon booklet; How can I involve the 
nonacttve linkeA?) li no tobk^ accAue, type it. N/A. . 

5. Attention: State PoAtneA 

hlameA oi othen^ who should receive a copy oi the n.epont 
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Basic Skills Instructional Improvement Program 

TIME/District, Principal 
Post-Session Questionnaire . 



Date: 



Location: 



Please rate each of the following by circling one of the five responses. 



1^ Expertise of the training staff 

2. Quality of presentation/training 

3. Quality of materials 

4. Relevance of content 

5. Likelihood that this program will 
work in your situation • 



1 
1 
1 
1 
1 



2 3 

2 3 

2 3 

2 3 

2 3 



4 

4 
4 
4 



5 
5 
5 
5 
5 



6. Wliat did you like best? 



7, How could the session be improved? 



8. Other conunents? 




TDIE/Dlstrict, Principal 



The statements listed below represent objectives of 
this session. Please indicate below whether you can: 



9. Define allocated time,, engagement rate, and student engaged time. 

10. Distinguish engaged from unengaged student behaviors in the ^ 
classroom. 

11. Record data on the engagement rate form and calculate engagement 
rate. 

12. Complete an allocated time log. ' 

13. Read and interpret- one of the time graphs '(find the expected 
achievement zone for a given amount of student engaged time). 

14. Establish an appropriate goal for student erigaged time. 

15. Establish the corresponding goals for allocated time and 
engagement rate (for a given student engaged time goal). 

16. Assist teachers in selecting appropriate strategies for increasing- 
allocated time and engagement rate. 

17. Help a teacher plan the implementation of a strategy to improve 
classroom management of students and management of instruction. 

18. Schedule the initial round of classroom observations. 

j[4U-'-^' Help teachers assess the impact of their changes on classroom 
practice, especially student engaged time. 




1 

1 

1 
1 

1 
1 



1 
1 



* 
* 

* 
2 

2 

2 

2 

2 



3 

3 

3 
3 

3 
3 

3 

3 

3 
3 



4 

4 

4 
4 

4 
4 

4 

4 

4 
4 



5 

5 

5 
5 

5 
5 

5 

5 

5 
5 



X/A 

N/A 

N/A 
N/A 

N/A 
N/A 

N/A 

N/A 



N/A 

N/A 

N/A 
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20- What specific points/questions need to be clarified in order for you to be able to use this training? 
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^ *Thi^^^ponse option inappropriate £or this item. 
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Basic Skills Instructional Improvement Progranr 
TIME IMPLEMENTATION /District, Principal 
Post-Session Questionnaire 



Date: 



Location: 



Please rate each of the following by circling one of the five responses. 



1. Expertise of the training staff 

2. Quality of presentation/training 

3. Quality of materials 

A. Relevance of content ^, 

5. Likelihood that this program will 
work in your situation 



6. What did you like best? 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



7, How coi^ld the session be improved? 



8. Other comments? 
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TIME IMVLKMENTATION/District , Principals 

The follovlng scatements . relate to planning for teacher * 
training and Implementation for student engaged time* 

Please. indicate the degree to which you believe 
you understand. ■ 

*9. Wlicn announcement of teacher training session(s) will be made and 
person (s) responsible for making announcement, 

10. Agenda and responsibilities for teacher training session. 

11. Intended outcomes of the teacher training workshop. ^ 

12. * What equipment will be needed for training, where it is located, 

and how to operate it. 

13. What materials are needed; when and how materials will be printed. 

lA. Process for establishing an initial observation schedule and who 
will do initial observations. 

15. Process to be used to alert teachers of the need to complete allocated 
time logs and pre-observation form on formal observation days. 

16. How implementation sessions/conferences will be achieved (individual 
teacher-principal conference, small group or grade level meeting, 
whole faculty or combination). 

17. How data will be shared with teachers, who and how comparisons with 
research will be made, and goals set. 

Kow strategies will be developed and/or shared with teachers. 



// J/ 



.0 -V 



m 



2 
2 
2 

2 
2 

2 

2 



. 3 

3 

3 
3 



A 
A 
A 

A 
A 



18. 

19. How inservice needs that are identified in selection phase will be 
handled. 

10. Wh.1t strategy principals will use in determining frequency, of 
classroom observation. 

n The frequency and/or level of "low" student engaged time data that 

" * should signal a more serious {complex) intervention strategy. 

22. I believe I can use the Time Leader's Guide. (Please check one) 

j j Or. my own , now 

[ I With further study 

r~l With some assistance 

I I Only with further instruction and assistance 

I I Even with further instruction, may not be able to do 

23. Whac specific questions or concerns do you wish to have answered before the teacher 
training session or before implementation? 



3 

3 
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Basic Skills Instructional Improvement Program , 
CONTENT/District, Principals 
Post-Session Questionnaire 



Date: Location: 



Please rate each of .the following by circling one of the five responses: 



1. ' Expertise of the training staff 12 3 4 5 

2. Quality of presentation/training 1 2 3 4 5 



3. Quality of materials 

4. Relevance of content 



1 2*3 4 5 

1 2 3 4 ,5 



5. Likelihood that this program will 1 2 3 . 4 5 - 

work in your situation 



6. What did you like best; 



7. How could the session be improved? 1 



8. Other comments? 



2 . 3 



5 
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CONTENT/D 



t, Principals 




The statements listed below represent objectives of 
this session. Please indicate below whether you can: 



9. Define prior learning, instructional overlap, and mastery. 

10. Use standardized achievement test data to project the classroom 
average percentile score that can be expected under ''normal" 
instruction. 

11. Use test data to identify a class's specific prior learning 
strengths and weaknesses. 

12. Find the overlap between the content taught and the content 
* measured by the test. 

13. Read an instructional overlap graph (find the expected achieve- 
ment zone for a given percent of instructional overlap). 

14. Examine initial instructional plan and decide whether resulting 
overlap is appropriate for a specific situation. 

15. Assist teachers in completing long-term instructional plans. 

16. Assist teachers in completing unit plans. 

17. Help teachers review their progress on the long-term plan and 
their s tudent s masrery"ol' trhe^oplx^ covered . 

18. Assist teachers in revising their long-term plans, if necessary. 




AT 



1 
1 



1 
1 

1 



2 

2 

2 
2 

2 

2 
2 

2 

2 



J' 



3 

3 

3 
3 

3 

3 
3 

3 

•3 



4 
4 

4 
4 

4 

4 

4 

4 
4 



5 
5 

5 

3 

5 

5 
5 

5 

5 



N'/A 
N/A 
N/A 

N/A 
N/A 

N/A 

N/A 
N/A 

N/A 

N/A 



o 



19. What speci 

p ■ 



fic points/questions need to be clarified in order for you, to be able to use this training^ 
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Basic Skills Instructional Improvement Program 
Planning for content implementation/District, principals 
Post-Session Questionnaire 



Date: Location: 



Please rate each of the following by circling one of the five responses. 



1. Expertise of the training staff 

2. Quality of prese^itation/ training 

3. Quality of materials 
A. Relevance of content 

5. Likelihood that this program will 
work in your situation 

6. What di'd you like best? 



7. ^How could the session be improved? 



-.o^" ^e^^^- ^^^^ 



1 2 3 _ 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



8. Other comments i 



V 
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for contt.t.t Lmpleutncation/Dlstrlct, pririclpa 



-he fcl-lowing Htate-nentr relate to content troiidng and 
ir".plementation. Please irdicatv. the degree to which you 



understand : 



V 




9. How to interpret the data to be used for identifying .prior 
learning strengths and weaknesses. 

10. How teachers will calculate an estimate of instructional overlap, 



11. How to code the assessmen 



t instruments to the curriculum guide. 



12. '^en and how the prior learning data and School Year Planning 
Guides will be made available to faculty. 

o 

13. Intended outcomes of the content training workshop. 

14 The roles and functions of teachers, principals, and district 
office staff in implementing the content variables. 

15. How principals will monitor teacher implementation of 
instructional plans. 



1 
1 



1 
1 



2 

2 



2 V 
2 



2 



3 



4 
4 



4 
4 



■ r. 



16. I believe I can 
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use the Content Leader's Guide (please check one) 



□ 
□ 
□ 
□ 



On my own, now 
With further study 
With some assistance 

Only with further instruction and assistance 



with further instruction, may not be able to do • 

17. .rhat specific questions or concerns do you -wish to have answered before the training session. 
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Basic Skills Instructional Improvement Program 
Planning for Teacher Orientation^ 1 

Post-f;ession Questionhaire > ^ 



Date: Location: 



Please rate each of the following by circling one of the five responses, 



1. ExperMfee of the training staff 

2. Quality of presentation/ training 

3. .Quality of materials 

4. Relevance of content 

5,. Likelihood that this program will 
work in your situation 

6. What did you like* best? 



7. How could, the session be improved? 



8. Other comments? 



1 .2 3 4-5 

,." 2 3 A 5 

1 2 3 4 5 

12 3 ^ 

l' -2 3 . 4 5 
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The following statements relate to the teacher orientation 
session. Please iikl^cate the degree to which you ^understand: 



1« The suggested agenda for the teacher orientation session. 

2. The objectives or expected outcomes. ^ 

3. Type and number of materials needed. , " 
4* Who will prepare lAaterials. 

5. Wlio, when, .and how the announcement will be made to the faculty 
about the purpose, time, and place of the teacher orientation 
session. 



■ A' A 
J-/ .^>^ /.^ 




Please indicate whether you can: 



6. Find the following materials for teacher orientation in my 
principal's guide 

a. Rationale, strategy, and content outlipe 

b. Handouts 

c* Transparency masters 

7. Present information from the orientation outline. 

8. Discuss the roles and functions of teachers, principals, and 
district staff. 

9. Answer likely questions about the basic skills instructional 
improvement program, 

\10. Follow up on individual teacher concerns. 



11. I believe the orientation session will be held on 



/ 



r.^' yP" ■^J' J^J'^ 

^ ■i' ^ r 




(date) at 



(location) . 



12. l' believe, the following people will be responsible for leading/assisting the teacher orientation session. 

Leading 

.Assisting 



13. What specific questions or concerns do you wish to have answered before the orientation? 
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Teacher Orientation 
Instructional Improvement 



Program 



Date Location 



Please indicate the degree to 

which you agree with the < a ^ <^ 

following statements. ^^^^^ o<^%'>^'' . o<'V'' 

1. The presentation gave me information 

that helps me understand some of the 1 2 3 4 '5 

activities planned for my school, ^, 

2. The materials used in the presentation ^ ^ 34, 5 
were effective • 

3. The improvement program described is 

relevant Lo my day-to-day classroom 1 2 3 4 • 5 

activities, 

4. The improvement program described is 

relevant to my school's instructional 12 3 4 5 

program, 

5. As a result of the pifesentation I am ^ 2 3 4 5 
interested In the improvement program. 

6. The improvement program will probably 12 3 4 5 
succeed in my school and classroom. 

\ . * > 

\ 

7. What do you like best about the program? 



8. , Was there information you believe should have been prestoted at the 
orientation session^ but wasn't? 
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9. Are there' concerns you have about the improvement program that you 
would like to discuss prior to program implementation? 



10. The teacher's guide you. were given at the orientation session is a 
developmental copy. Do' you Kave specific suggestions for improving 
it? (For example, should it discuss or describe something that is 
not now discussed? Does it raise some concerns that it does not 
address?) 



\ 




Please return to . ^ ^ 
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Date : 



Basic Skills Instructional Improvement Program 
TIME/Teachers 

Post-Session Questionnaire 



Location; 



r 



Please rate each of the following, by circling one of the five responses, 



1. Expertise of the training staff 

2. Quality of presentation/training 

3. Quality of materials 

4. Relevance of content 

5. Likelihood Lliat this program will 
work in your sitiiation 



ro° 0^ . ^ 



1 

1 
1 
1 

1 



2 3 

2 3 

2 3 

2 3 

2 3 



4 
4 
4 
4 
4 



5 
5 
5 
5 
5 



6. What did you like best? 



7, How could the session be improved? 



8. Other comments? 
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T^ME/Teachers 



I 



int 3*:c.t:tsr,3r.cs listed Dciow. represant cbjectivt 



of 



chis Session. Please indicace belov; whether you can: 



1 5 5 



9. 


Define allocated time, engagement rate, and student engaged time. 


1 


* 


*> 


4 


5 


N/A- 


10. 


Distinguish engaged from unengaged studeat behaviors in the ' 1 
classroom. • 


1 


* 


3 


4' 


> 5 


, N/A 


11. 


Recor^J data on the engagement rate form a' ' calculate engagement 
rate. * ' . . \ 


1 


* 


3 


4 


5 


. N/A 


12. 


Complete an allocated time log. 1 


, 1 ^ 


* 


3 


4 


. 5 


N/A 


13. 


Read and interpret one of the time graphs (find the expected \ 
achievement zone for a given amount of student engaged time). \ 


1 


* 


3. 


4 


5 


N/A 


14. 


Establish an appropriate goal for student engaged time. 


1 


2 


3 


4 " 


5 


N/A 


15. 


Establish the corresponding goals for allocated time and 
epgcga.ViCnt rate- '(ft)r a given student engaged, time goal). 


1 


2 


3 


4; 


5 


N/A 


16, 


Select appropriate strategies for increasing allocated time 
and engagement rate. 


1 


2 


3 


4 


5 


N/A 


17. 


Develop a plan for implementation of a strategy to improve • 
classroom management of students or management of instruction. 


1 


2 


3 


4 


5 


N/A 


18. 


A-ssess the impact of the strategies on classroom practice,, 
especially student engaged time. 


1 


2 


3 


4 


5 


N/A 



15 



19. Wlaat specific points/questions need co be clarified in order for you to be able to 'use this training? 



*This response option inappropriate for this item. 
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Basic Skills Instructional Improvement Program 

CONTENT/Teachers 
Post-Session Questionnaire 



Date: 



Location: 



Please rate each of the following by circling one of the five responses. 



.V- 0°" .^-jf ^o-^ ^ ^ 

b^^^ ^^^-^ " .oo^ 



].. Expertise of the training staff < 

2. Quality of presentation/training 

3. Quality of mat'erials 

A. Relevance of content 

5. Likelihood that this program will 
work in your situation 



1 
1 
1 
1 
1 



2 
2 
2 
2 
2 



3 
3 
3 
3 
3 



A 
4 
4 
4 

'4 



5 
5 
5 
5 
5 



6. ' Wliat did you like best? 



7, How could the session be improved? 



8. Other comments? 
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0 

COXTENT/Teachers 



The sc^cebents listed below represent objectives of * • ^ 

chis session. Please indicate below whether you can: v^- 



DO 
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9i Define prior le'^rning, instructional overlap j and mastery , 



instruction. 

11, Use test data to identify a class's specific prior learning 
strengths and weaknesses. ^ 

12. Find the overlap between the content taught and the content 
^ measured by> the test. 



14.. Examine initial instructional plan and decide whether resulting 
overlap is appropriate -for a specific situation. 

15. Complete a long-term "instructional plan. 

16. Complete a unit plan. 

17. Review progress on the long-term plan and your students' ' ^ 
mastery of the topic covered. . ^ ^ 

IB. Develop revised long-term plans, if necessary. 12 3 4 5 ^ 

What specific points/questions need to be clarified in order for you to be able. to use this training? 





1 2 3 4 5 NVA 



10. Use standardized, achievetfient test data to project the classroom 

average percentile score that can be expected under "normal" 1 2 3 4 5 ^ N/A 



1 2 3 4 5 • N/A 

1 2 3 4 5 N/A 



.13. Read an * instructional overlap graph (find the expected achieve- 

■ment zone for a given percerit of instructional overlap). 1 2,, 3 4 5 N/A 



1 2 3 4 5 N/A 

12 3 4 5 , N/A 

12 3 4 * 5 , N/A 



N/A 
N/A 
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Time Begin: 
Date: 

Respondent: 
Interviewer: 

BSC ADMINISTRATOR/TRAINER INTERVIEW 



\^at do you see as the*^ major goals and objectives of this approach 1 



A. How do these [^oals or objectives fit in with your view of 
the scl\pol district'' s *(or your particular scliool^s) goals 
and objectives? * 



l6o 
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To what extent do you think that the approach will succeed m your 
school district (or your particular school) in achieving its mam 
objectives? (PROBE FOR EFFECTIVENESS RE IMPROVING STUDENT ACHIEVE- 
MENT, CLASSROOM INSTRUCTION, AND INSTRUCTIONAI. LEADERSHIP) 



\ 



Based on your understanding of the approach thus far, what would yo 
say are the approaches' most important features? 



■3 
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In August and September what did you think your main rolii in implementing 
the. approach would bo? (PROBE FOR WHAT R WOULD ACTUALLY RE DO 1 NO IN 
IMPLEMENTING THE APPROACH) , . 



Were there any major problems that made it difficult for you to 
perform this role as you viewed it? (IF YES, IDENTIFY PROBLEMS) 



/ 

/ 



Now I'm interested in getting your specific reactions to the orlcmLation 
and initial (technical) training you received «rom RBS on the use of the 
approach for this year's implementation. Wliat are the. most important 
things you feel you have learned from this training? (PROBE FOR SPECinC 
KNOUTLEDGE AND SKILLS R MAY HAVE DEVELOPED) . . 
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(IF R IS A RESOURCE TEACHER, SKIP TO Q. 10; 
OTHERWISE ASK Q'S 6, 7, -8, AND 9) 



6. How adequately do. you think you have been prepared to successfully 
implement the approach? 



In what specific areas do you 'think you were not adequately 
, prepared? 



7. -Have teachers in your school (district) received any inservice training 
on this project up to this point? (IF YES, ASK R TO DESCRIBE TRAINING) 
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Did you play any part in providing this training to teachers? (IF YES 
ASK ABOUT BOTH CONTENT AND TIME TRAINING; IF NO, SKIP TO Q, 10) 



A, Content 



B. Time 



Did you have any difficulties delivering any parts of the training? 
(PROBE FOR BOTH CONTENT AND TIME TRAINING) 



A. Content 



B. Time 
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(NOTE: 



S3 

RESOURCE TEACHERS AND COORDINATORS MAY NOT 
BE ABLE TO ANSWER Q'S 10 AND 11) 



Are there any aspects of the approach that you feel have been difficult 
for teachers to implement? 



How would you assess the teachers' overall reactions to the project 
thus far? 
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12. What suggestions do you have for RBS (or for the district) with respect 
to providing follow-up assistance to you on this project? 



/ 

V 

y 



13. Do you have aijy other comments, questions, or concerns that you would 
like to share jat this point about the project? 

\ . • ' ' ' 



\ 




\ 

\ 

\ 

\ 

. \ ; . \ 

' " ■ \ 

i ' \ 

\ 

(TIME END): ' 




\ 



PRINCIPAL/ DISTRICT CONFERENCE 



Usin- the school suinmar/ data, the principal and district leadership 
together, answer the following questions, agree on the opportunities 
for improvemenc, plan and take necessary action chat will lead to 
improved instruction and greater student achievement. 



1, Based upon entering achieveme-nt data, which classes offer 
greatest opportunity for improvement this year in reading? 
nath? 



2. Have teachers identified strengths and weaknesses on their Quarterly 
Plan for reading? math? What are the weakest areas? 



3. How good is the match between Quarterly Plans and the Guide in 
reading? math? Wliat are the exceptions? 



:9 

4. How well are teachers maintaining the pace set by their quarterly 
plans for reading? math? Where is special help needed? 
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Principal/District Conference— Continued 

5. How successful Is the daily learning in reading? math? 'Where is 
special help needed? . 



6. What is the mastery rate in reading? math? Where is special help 
needed? 



7. How well is .time being used for reading? math? Where is special 
help needed? 
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Date: 



Interviewer: ' 

Interviewee: 

BSC District Level Staff Interview 

End of Year 

I'd like to get some of your thoughts about the Achievement Directed 
Leadership Program with which you have been involved this year. Td 
appreciate it if you would answer the following questions: 

1. What percent of your time was spent doing tasks related to this program? 



2. With what tasks were you involved? 

(Probe for planning implementation, planning teacher inservice training, 
data collection e.g;, scanning classes, conductingi'principal/teacher 
conferences, helping teachers plan basic skills instruction, attending 
principal seminars) ' 



Generally, do you' think this program v^as effective? 

(Effective in terms of ral'sing student achievement scores and improving 

classroom instruction.) If not, why not? 



4. Can you think of any problems that made this program difficult to 
implement? 
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What did you like best about the program? 



What did you like least about the program? 



Do you have any suggestions for making the program more successful 
next year? 



Do you have any other comments, questions or concerns about the 
program that you would like to share? 
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Date: 
Interviewer: 
Interviewee: 

BSC Principal Interview 
End of Year 



I'd like to get SjOme of your thoughts about the RBS SchooMmprovement/ 
Achievement Directed Leadership program with which you have been .involved 
this year. I'd appreciate it if you would answer the following questions? 

1. How important is the improvement in basic skills instruction to you? 
(Probe: Is it one of the primary goals you have for your school?) 



2. How important do you think the improvement in basic skills instruction 
is to parents who have students in this school? 



3. What was your impression of/ reaction to your students' 1982 achievement 
scores? ' ■ . 
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Overall, do you think this program was effective in your school? 
(Effective in terms of raising student achievement sco^^es and 
improving classroom instruction) How? 



Were there any problems that made it difficult for you to implement 
this program n*n your school? (If yes, identify) 



Did you train teachers? If yes, did you feel comfortable training 
teachers in time? If not, why not? 



In content? If not, why not? 



Did training include success/mastery review? 



i 7p 
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Do you think the program helped ybu become more successfuU^helping 
teachers plan basic skills instruction? Why/why not? 



Do you .think that the program helped you become a more- effective' 
supervisor of your school's basic skills program? Why/why not? 



What did you like best about the program? Le[|st! 



How did district level staff/DOI help you implement this program in 
your school?. If. so, how? In what ways did this differ from past 
emphasis? . 
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11. Did you find your conferences with the district level administrator 
helpful? (If not, why not?) 



12. Do you have any suggestions for makirig the^program more successful 
next year? . ^ 



13. Do you have any other comments, questions or concerns about the 
^.program that you would like to share? 
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Date: ^ 



Interviewer: 
Interviewee: 



BSC Teacher Interview 
End of Year 

« I'd like toiget some of your thoughts abcut'^the instructional improvement program 
.in which yoq have been involved and I'd appreciate it if you would answer these 
questions. |Your answers will be confidential . 

, 1 . ' What .do iou see as the major goals of this basic skills improvement program? 



ERIC 



2/ Have these goals influenced the goals you set for your classes? 



3. Did you receive enough training to sufficiently understand the program? 
If not, what didn't you fully understand? 
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Were there any problems that made it difficult for you to implement this 
program? (If yes, identify) 

PROB.E- Did the classroom scans/observations for tima-on-task present 
any problems? Did the Quarterly Topic Plans? Did the AT Log? 
Did the Curriculum Mapping Guide? 



) 

What instructional changes did you make as a result of your participation 
the program? (i.e., What did you do differently in planning for, managing 
or instructing your class?) ' 



What did you like best atpout the program? 
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7. What did you like least? 



8. How did the principal work with you in implementing this program? 



9. Did you find the' Principal/Teacher Conferences helpful? 
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10, Generally, do ydj think the program was effective in^your school? 
('Effective in terms of raising student achievement and improving 
classroom instrudt;lon) If not, why not. 



4 



11. Do you have any suggestions for making the program more successful next 
year? 

PROBE: Is there anything the principal or district could dd to help 
teachers implement the program better? 



12.- Do you have any other comments, questions or concerns about the program 
that you would like to share? 
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PRINCIPAL QUESTIONNAIRE 




^enri 



Dear Principal, 



This 'year, your district has been par ticipatirvg in a program designed to improve : 
instructional le.adersh.ip in the basic skills. Your cooperation in completing this 
questionnaire will be of great help in evaluating the success of the program. 
Please answer the questions by checking the appropriate box or filling in the 
appropriate blank. Thank you for your assistance. 

ALL QUESTIONS REFER ONLY TO CURRENT SCHOOL YEAR. 




Please estimate the percent of your time that was spent planning and implementing 

the Achievement Directed Leadership Program for your school* % 

How much of this time was spent • 

attending principal seminars 

planning for implementation 

planning for teacher inservice training 

training teachers 

helping teachers plan instruction 



supervising teachers and holding principal/teacher 
conferences 

attending superintendent/principal conferences 



Did you conduct any inservice training in. the area of: 
student engaged time? 

the overlap between the content you actually taught 
and the cyear-end standardized achievement test? 

attention to students' prior learning deficiencies? 

student success rate? 

Student mastery of basic skills content? 
the effectiveness of review? 



^ES 
[ ] 

[ ] 

[ ] 

\ ] 

[ ] 

[ ] 



-WO 

[ J 

[ ] 

[ ] 

[ ] 

[. ] 

[ ] 
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This year , YES^ NO 

Did central office staff memb(»rs spend more Lime 

with you or your teachers planning and supporting [ ] [ ] 

basic fikils instruction? / 

In your .opinion, did the overall ef fectiveness ;of the 
sch ool district in supporting basic skills instruction [ ] [ ] 

Increase? 
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DON ' T 

KNOW 



[ ] 



[ ] 



Did you spend more time with teachers planning and 
helping them with basic skills instruction 

In your opinion, did the overall ef fectivene55s of 
your schcKU in providing basic skills instruction 



YES 



In conferences with district level administrators, C 1 

did you discuss: 

[ ] 
[ ] 

^ ■ [ ] 

[ ] 
[ ] 

^ - [- ] 



student engaged time? 
content coverage? 
prior learning? 
success rate? 
mastery? 
review?- 



This year , 

Were your teachers successful in reaching or maintaining levels 
of student engaged time associated with high achievement? 

In general did your teachers actually cover more of basic skills 
subject matter that thev had in previour years? 

Were vour -.eachers able to give more attention to students' 
prior learning deficiencies than in past years? 

Would you say- that your teachers' use of content review was 
more extensive ar-l/or systematic than in past years. _ 

As a result of the instructional improvement program, did your 
teachers, change their method of instructional planning? 

As' a result of the instructional improvement prograi^, did any 
of your teachers change their classroom management procedures. 

As a rosuU of the iustructionaj improvement program, did any 
of y.uu- teacher.s change Lheir techniques in the basic skills 
areas;' 

ilvornl"), would vou sav that vour teacliers ^ere more effective 
in providing basic skills instruction Co students? 
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These questions refer only to the Achievement Directed 
Leadership program: 

Have you thought about or discussed ways of improving the 
program? 

Have you thought about the impact of the program on your 
school? 

Have you thought about expanding the techniques of the 
program to areas other than the basic skills? 

Did you actually expand the program's techniques in your 
school? . 

Would you say you were generally successful in doing the 
activities required by the program? 

Would you say that, for your school, the program went 
smoothly? 

Have you shared your tdeas about the program with 
other principals? 

What is your overall reaction to the basic skills 
instructional improvement program in which your school 
has been participating this year? 



Very positive 
Positive 
Neutral 
Negative 
Very negative 



YES 

[ ] 

[ \ 

[ ] 

[ ] 

[ ] 
[ ] 
[ J 
[ ] 

[ ] 
[ ] 
[ ] 
[ ] 
[ ] 



NO 



['lease feel free to use the back of Che questionnaire to, comment at length 
about the program. ■ 




.WOflflL 
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TKACHER QUESTIONNAIRE 



Dear Teacher, 

This year, your district has been participating in a program designed to improve 
instruction in the basic skills. Your cooperation in completing this question- 
tiairc will he of great help in evaluating the success of the program. Please ^ 
answer the questions by checking the appropriate box or filling in the appropriate 
blank. Thank you for your assistance. 





ATT mTiTQTTnNc; BfrPFB nm.Y TO CURRENT SCHOOL YEAR . 

Did you look at research findings (on time and/or content 
graphs, etc.) to help y6u set improvement goals for your 
classrooms? 

Did you use your classroom data and research to help you 
improve your teaching? 

If vou answered ^ to the preceding question, did you 
actually do anything to improve in the area of: 

student engaged time? -'i- ^ 

the overlapV-between the content you actually taught 
and the year -end standardized achievement tests? 

remediation of students' prior learning 
deficiencies ? 

student success rate? ^ 
student mastery of basic skills content? 
review of basic skills subject matter? 
Did you receive any inservice training in the area of: 
student engaged time? 

the overlap between the content you actually taught 
and the year-end standardized achievement test? 

• attention to students' prior learning deficiencies?, 
student success rate? 

student mastery of basic skills content? 
the effectiveness of review? 



YES NO 



[ ] 



C ] 
C ] 

[ 

C ] 

C ] 
C ] 
C ] 

C ] 

.[ ,] 

C ] 
C ] 
C ] 
C ] 
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Did you receive any assistance (other than inservice - 
training) related to: ■ ' 

•Student engaged time? 

content coverage? 

attention to students' prior learning? 

•J 

student success rate? ? 
student mastery of basic skills? 
review of basic skills subject matter? 



YES NO 

[ ] . [ ] 

[ ] [ ] 

c ] [ ] 

[3 [ ] 

[■] t] 

[ ] .[ 1 



c DON'T 
Y ES NO KNOW 



increase 



This Jp^j: ■, ■ 

Did your principal spend more time with you planning |- ■] [ ] [ ] 

and helping you with your basic skills instruction? 

In your opinion, did the overall effectiveness of [ ] [: ] [ ] 

y our school in providing basic skills instruction 



[ ] [ ] n 



Did central office staff members spend more time plan- 
ning and supporting basic skills instruction? . ■ 

In vour opinion, did' the overall effectiveness of the |- ] [ ] [ ] 

school di strict in supporting basic skills instruction 
increase? ' . • 

How many times was your classroom observed for the 

purpose of collecting data for student engaged time? ' (number) ' 
How many times did you have a formal conference as 

■part of the basic skills instructional improvement (number) 
program? ■ „ 
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tn these conferences, did you discuss: 
student engaged time? 
content ^average? 
prior learning? 
success rate? 

> 

Tuastery? 
review? 



YES 
ALWAYS 



YES 
SOMETIMES 



NEVER 



NOT 
SURE 



This year , 



YES 



NO 



Were you successful in reaching or mainfarning"levels^ 

of student enf^aged time associated with high achievement? 

Did you actually cover the amount of basic skills 
subject matter that you had planned to cover? 

Were you able tg give -more attention to students' 
prior learning deficiencies' than in past years? 

Was your students' daily success rate appropriate? 

Did more, of your students master more basic skills 
objectives than in past years? 

Was your use of r eview more systematic than in 
past years? 

Did you change your method of instructional planning? 

Did you change your class. ^ management procedures? 

Did you change your teaching techniques in the basic 
skills areas? , . 

Overall, were you more effec^iive in providing basic 
skills instruction to your students? 
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YES 



NO 



These. questions refer only to the basic skills instructional 
improvement program: ^' 

Have you thought about or discussed ways of improving the 
program? 

Have you thought about the impact of the program on Vo^^ 
classroom and school? 

Have you thought about expanding your use of the techniques 
of the program to areas other than the basic skills? 

Did you actually expand your use of the program's 
techniques in your classroom? 

Would you say you were generally successful in doing 
the activities required by the program? 

-Would you say that,, for you, ^the program went smoothly? 

Have you shared your ideas about the program with other 
teachers? 

What is your .overall reaction to the basic skills 
'instructional improvement program in which your school 
has been participating this year? 

Very positive 

? Positive 

Neutral 
Negative 
Very negative 



[ ], 

[ ] 

[ ], 

[ ] 

[ ] 

[ ] 

[ ] 



[ ] 
I ] 
[ ] 
[ ] 
[ ] 



[ ] 
[ ] 
[ 1 

'[ J ' 

[ ] 
[ ] 

[ ] 



Please feel free to use the back of the questionnaire to comment at length 
about the program. 
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Titne Begin: 
Respondent: 
Interviewer: 
Date: 



BSC PRINCIPAL INTERVJEW 
(Participatory Supervision) 



1. How many conferences hdve you held up to now? 



2. Approximately with what percentage of the teachers have you had conferences? 



3. How long has a typical conference been? 



4. Where were conferences primarily held? 



minutes 



Principal 's office 
Teacher's classroom 
Other (SPECIFY) 



"5. During what part of the day were conferences held? (CHECK AS MANY AS 

'apply) 

Lunch time 

Planning period 



Before or after school 
hours 

Other (SPECIFY): 
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6. WeV there any others besides you and the teacher present at any of 
the conferences? . 

Yes(ASK Q.'s 6A-6D) 

No (60 TO Q. 7) 
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A. Who? (ASK FOR NAME AND TITLE) 



B. How did he/she participate in the conferences? 



C. Did he/she .participate in most or all the conferences? 



Yes 
No 



D. How was it decided that he/she would participate in the 
conferences? 



i] 
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7., Did you do. anything to prepare teachers for the conferences before they, 
*'were act^^ally held? „ 



Yes (ASK Q. 7A) 
No (GO TO .Q. 8) 



A. What did you do and why? 



8. What did you say to teachers to begin your conferences? 



9. could you briefly describe a typical conference^- how it w^^^^ 
« MOST ATTENTION, JlfC OR CONI^ . " 
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H0« v,ould you Characterize the o-ran tone^o. affect of your conferences 
(PROBE FOR TRUST LEVEL, SPIRIT OF COLLABORATION) 



' (ASK Q 's 11-13 IF R USED TE/?CHER/ PRINCIPAL CONFERENCE (T/P) FORM. 
, OTHERWISE SKfP TO Q. 14) / 

,1 «ere there' any questions on the T/p'Vorn, that were xi>ffi cult to.under- 
■ stand? fSHOW R T/P FORM) 

, _____ Yes' (ASK Q. HA) 
' . No (GO TO Q. 12) 



A. Which questi-ons were difficult and why i 



data; achieverrfent data; mapping guide; ct,c. }i 



Yes (ASK Q. .12A) 

^ " . ' No ('^0 TO Q. 13). 
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A. What information did you not have? (PROBE REASON(S) 
WHY NOT) 



/ 



1 ' 



i 



were there any pUl ems using or interpreting the information you had?. 

, / _^ Yes (YES Q. 13A) 

■ No (GO TO Q. 14) 



r' 

A. What problems? 



«ho conducted the classroom observations?-. (CHECK AS MANV AS APPLY) 

- ^ Principal 

Assistant Principal . 

'" Teacher 

District Office Staff 

Other (SPECIFY): 
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5. Describe how the classroom observations were done and ^ny problems you 
had? 



i5? 



16. AS a result of conferences: . 



i ^ 



. / 

/ 

/ 



A WprP anv sbecific opportunities or needs for instructional 
^rroveyt ldentified? (IF YES, ASK R TO GIVE ONE SALIENT 
EXAMPLE}' 



B Were any tentative goals set for improving instruction? 
' (IF YES, ASK R TO GIVE ONE SALIENT EXAMPLE) 



y) *^ ■ , '» 
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C Were any specific strategies selected for reaching -goals 
tZt were set? (IF YES, ASK R TO GIVE ONE SALIENT EXAMPLE) 



D Were any plans developed for implementing strategies for 
iSJroving instruction? (IF YES. ASK R TO GIVE ONE SALIENT 
.EXAMPLE) 



E. Were any plans developed for determining whether teachers 
are in fact using the improvement strategies that were 
selected? (IF YES. ASK R TO GIVE ONE SALIENT EXAMPLE) 



s 
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17. Ir. your opinion, were conferences beneficial or not b^^fj j^j^^ j" J^^f 
of fostering improvement in basic skills instruction? (PROBE REASONS) 



A. Were conferences beneficial in any other way? 



18. Do you have any suggestions for improving the conferences, for example, 
in terms of: 

A The procedures for getting the necessary information for 
conferences?^ (IF YES, EXPLAIN) 
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B'. The content to be covered in the conference? (IF YES, 
EXPLAIN) 



C The T/P Conference Form or any other forms designed to 
'facilitate the conference? (IF YES, EXPLAIN) 



D. Anything else? 
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Have you had opportunities Vet.to see If any i-t^^j"-:,-^™''^"*"' 
strategies Have actually been implemented in the classroom. 

Yes (ASK Q.'s 19A-19C) 

No (60 TO Q. 20) 



A. What have you (gr anyone, else) done "for follow-up? 



•1 



B. How many classrooms have -been followed-up? 



C. What have been the results of this follow-up? 



r 
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20 Have you received any assistance or support from the distnict's 
'central office with regard to implementing this supervision , 
process? 

. Yes (ASK'Q. 20A) 
No (GO TO Q. 21) 



A. What kind of support? 



21. Are there any other comments you would like to make about the con 
ferences or about the program in general? 
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■2. Vie would like to Identify about 5-10 teachers j"3^y°",J,=*-l,S5?E«r " ' 
•could talk about th6 conferences. (SHOW R ^1^^ BSC TEALHtK initKvi , 



\ crade ■ ^"^^"^^ 

Name of Teacher \X Name .of Teacher Level 




\ 
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Grade ' 

Quarter (circle) 
1 2 3 4 



TEACHER/PRINGIPAL CONFERENCE 



Teacher and principal together answer the following questions, ^gx^^on 
the opportunities for improvement and plan necessary actions that will 
lead to improved student achievement. 



The enterxng achievement level of thia. class 
is: 

a, 55th percentile or above 

46th to 54th percentile 
c. 45th percentile or below 

Comment /Opportunity/Plajfi: 



Are the skills-related strengths and weaknesses 
indicated on the Quarterly Topic Plan? . 

'Comment/Opportunity/Plan: 



3* 



Of the topics listed in the. Basic Skills 
Management Guide the Quarterly Topic 
Plan indicated; 

a. 80 - 100% of Guide covered 

b* 60 - 79% of Guide covered 

c. Less than 60% of Guide covered . 

Commen t /Op po r tuni t y / P 1 an 
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Reading/ 
, Language Arts 



la □ 



lb 
lo 



2 Yes 
2 No 



3a . 

3b 

3c 



Math 



la 
lb 
Ic 



2 Yes 
2 No 



3a □ 
3b □ 

3cn 



0 , 



Teacher /Principal Conference — Continued 



4. The class is covering the Quarterly Topic 
Plan at the following "^pace: 

a. More than 4 days ahead of plan 

b. +4 days, from planned 

c. More than 4 days behind- plan 



C ommen t / Opp o r tuni t y / P 1 an : 



Reading/ 
Language Arts 



4a 
4b 
4c 



5.^ Students' daily work in this quarter indicated: 

a. 80 - 100% are successful most of time 

b. r ^50 - 79% are successful most of tiifie 

c. Less than 50% are successful most of time 

Comment/Opportunity/Plan: 



5? 
5b 
5c 



The percent of students mastering skills at the planned rate 
during tl^e quarter is: . 



a. 80 - 100% of Glass 

b. 50 - 79% of class 

c. Less than 50% of class 

Corament/Opportunity/Plan: 



6a 
6b 
6c 
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Teacher /Principal Conference — Continued 



7. The average student engaged tima is: . 

' The average student engaged time predicts* 
end of year achievement: ' 



a. above expected 
b , at expec'ted . 
c. below expected 

Comment /Opportianity /Plan 



Reading/ 
Language Arts 



mm. 



7a □ 
■76 □ 




8. The average . allocated -^tinre is: 

a. above district minimum 

b. at district minimum 

• c. below district minimum 



mxn, 



8a 
8b 
8c 



Comment / Opportunity/Plan 



9 . The average engagem^t rate is : 



a. 70 - 90% ^a 
■ b . 50 - 697. ■ ' 9b 

c. Below 50% 9c [_J 
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Teacher /Principal Conference— Continued 



Reading/ 
Language Arts 



10* Opportunity for improvement in engagement 
^rate is greatest in the following ^category : 



Comment /Opportunity /Plan : 



M □ 



s 

D 
U 
0 
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est CA;r_ 

Teacher # 



Form 



' ■ CONTE^IT FORM 
Level 14 



« 

Dates Cample ted 



Page of 2- 
. ^ , ■ . ; . V 

Subtest Reading Vocabulary 

few - 

Notes — ^1*. 



171 Expected Coverage 



X Covered 



i 

1. 
I 

1 

ER 


if * • 

^£ontent Mea^'sured by Test Uem 


1 Percent o'f Class 1 


Code 
No. 


U 


1 n 
1 U 


1,1- 


31-' 

70 


71- 
QO 


■ { 
Ql-1 

100 


SAME 'MEANING (SYNONYMS) . ■ 

The student silently reads a^phrase and four (4) 
•possible cho:lc.es. The student se'lect^^ the word that means' 
f'l-irt o'lmn nc t-np nn (IpT^l i Tied wt)rd in the phrase. * 

m . ^ » 

1. * 






- 








\ 


2 . . ^ 






























^ — ^ 

U \f ^^^^ — — . ■ — " ; 
















3. ... — ^ 






6 








! 


6. ' 














— 1 


: 7: ■ ■ : . ^— '- 












■ 


, 


* 8. • • 














. 


» . * t 

q. • _ ^ 


v> 






a 






. 


10. ■ 
















11. -5 














12. ' . » - 
















13. - 1-^ r— 
















14. _ 

















L5. , ■ ■ ' 
















-^^-^ — 














^ 


17. • . 
















^ 18. . ^ 




J 













































CO 12/11/80 ' . qONTENT FORM (cont'd)* 

Level Subtes t Reading vocabulary [7] Expected Coverage 



Page JL of A 
. [X] Covered 




Content Measured by Test Item 



OPPOSITE. MEANING (ANTONYMS) 



The -student silently reads a phrase, aiVd four (4) 
possible choices. The student selects the one word that means 
'the opposite of the und.erlined word in the phrase. 



21, 



22 



23-. 



24. 



25-. 



MULTIPLE MEANING 

The -student silently reads ^a definition and three (3) 
senterices below it. The stiid^snt selects the sentence in ^which 
the underlined word is used as described ^^.n the definition. 



l-:XMff'Ll'::.' poBt: a piece"of wood ^ 

* o Kd went t'o the post office, 
.•'o She hung her coat on the post . 
o The teacher wil,l pos. t the notice. 



fr", 



26. 



27.. 
28. 



29_._ 
30. 



Code 
No. 



ERIC 



. / 1^6 



Percent of Class 



0 



10 



n- 

30 



31- 

70 



71- 
90 



91- 
100 



I 



CO 12/11/80 

St CAT_ 

Teacher H 



" ° CDNTEMT form" . .. P^Se L of 1- 

Form c . Level ' li Subtest jieadni^ComErehc.^^^^ . 

Dates Comple.ted , 



[7] Expected Coverage 



Notes 



X Covered 



Content Measured by Test Item ' 



LITERAL COMPREHENSION 

The student silently reads a s.xection and the questions 
.that follaw it. The student selects the word or phrase from 
•four (4) possible choices, that be^ answers the question. 

RECALL OF FACTS 
31. Narrative - Wlio? 



Code 
No. 



Percent of Class 



1- 

10 



11- 

30 



31- 
70 



71- 
90 



100 




CO i2/.l.L/80 

Reading 

Level . ^ Subtest comprehension 



Page J*_of 5_ 

CONTENT FORM (cont'd) - 

[7] Expected Coverage [X] Covered 




Content Measured by Test Item 



INTbRPRETlVE COMPREHENSION 
"INFERRED MEANING 
34. .Narrative il- main idea 
50._ 

61 J' 

51'. " . ' 



II II 



II II 



conclusion 



60j_ 
•53. 
6 3: 
65. 



" - cause/effect 



CHARACTER ANALYSIS 
33. Narrati ve - feeling 
40_, . " . 

3_2_. 

3_8._ 

39.. 

35. 



- motive 



trait 



57_._ 

54_. 

55.' 



at titude 



- compare characters-. 
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Percent of Class 



Code 
No. 



10 



2(<r 
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n- 

30 



31- 
70 



71- 
90 



91- 

ino 





CO 12/11/80 

Reading 

Vel 1^ Subtest comprehension 



Page s s 

CONTENT FORM (cont'd) 

[7] Expected' Coverage [X] Covered 



Content Measured by Test Item 



Pi CURAT [ VE LAflGVArJi 



Code 
No. 



Percent of Class 



66. Sent enre - simile 

-<! — — — ' 

68. . 



4 7^ ^^_^£LJZ^!}l^^iS^.2JL 

69 > Sentence - metaphor 
57^ " - hyperbofe 



70. 



Poetry - personification 



CRITICAL COMPREHENSION 
mAL/mRF.AL KLEMKNTS 

The student reads three (3) sentences and selects 
the one sentence that tells about something that could really 
happen. " - 

AUTHOR ATTITUDE/POSITION 

4 1. Report - fact/opinion 

42. " ^ " " ; 



1- 
10 



n- 

30 



31- 

70 



71- 

90 



91- 

100 



43. 



45. 



44. 



compare viewpoints 
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CO 

11/10/8'^ 


1 State // 


SUMMARY 


Teacher / 


INSTRlKVnONAh OVKRLAP 
j Snh'^t»,ct . , 


State 
District 


1 District 
S School // 


// 


Grade 


Test 


Srhool 

Teacher 




Date(s) 


Form . Level _ 




DIRl.CTIONS ' 

Count the slashes (/) in each 
column for each page of the 
Content Form and record the 
counts In the appropriate 
spaces. 

Add each colunu^ to find the 
total number of slashes in . 
each coluw- 

» Add Total row to find total 
number of slashes on all 
pages and record below. 

> Find total number of test 
items and record below. 

I If total number of slashes 
is not equal to total number 
of test Items, check column 
totals and counting of 
slashes. 

% Multiply eacli total by the 
indicated midpoint to get a 
product for each percent of 
class . 

• Record the sum of the product 
row in the last box. This sum 
is the number of items covered 
by the average student. 

• Divide the average raw score 
by the number of items on 
the test to get thB percent 
of overlap for expected 
coverage . 

Total Number of Slashes 



l-.XRKCTKD, C0VKKA(;K (/) 




PAGE NO.' 0 



1-10 



PERCKNT OF CI./\SS 
11-30 31-70 71-^0 



10. 
11. 



12. 



13. 



lA. 



91-100 



TOTAJ. 



MIDPOINT 



PKODUCT 



.05 



.20 



.50 



.80 



.95 



AVERAGE RAW SCORE 




Total Number of Test Items 

Average Raw Score 
Expected Coverage Overlap - Total NoV of Test It 



Items 
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..r.it.»- 


— 


j 




S s I r i c t 














State # 


Ttviv her 






District # 





SUMMARY SHEET FOR TIME 



School # 



■Teacher # 



Grade 
Year 



f^f^^s 2H7 




tU'ADINu/IANCUAGE ARTS 



00 
00 



MATfl 



1 

t 

-Date } Coder // 


Van of 
Period 


Engagement 
Rate 


Allocated 
Time 


Student 
Engaged 
Time 


o 51 \.r P <i T i 'Ti • 


i 




■ 












1 » ■ ■ 
































% 



























Co-ier it 


l\'irt of : 
Period 


Engagement 
Rate 


Allocated 
Time 


Student 
Engaged 
Time 


Average 
■ Student 
Engaged '/imo 


— -] — ■ 

1 












1 . 












1 

! 












1 




r — 

■ -. . 









































1. 

2. 
3. 
4, 



Student Engaged Tine « Allocated Time x Engagement Rate 
DIRECTIONS 

Use this form to summarize data throughout the school year. 
Fill in the information at the top of the form^ 

Record the dateCs) of observation. One line on the form is used for each day. 
Record the total allocated time (in minutes) from the Allocated Time Log for 

ScSrrtircoder numbers, parts of the period observed, and average engagement 
rate for all observations made on that date. • 
Find the student engaged time (allocated time x engagement rate). 
Find' the average student engaged time for all observations by adding all ot 
the entries in the student* engaged time column aijd dividing by the number of 
entries. ^{^S 
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Appendix D 



Development of Summary Indices Describing 
Level of Program Implementation 




Development of Summary Indices Describing 
Level of Program Implementation 

A large amount of Information regarding Implementation of the 
Achievement Directed Leadership approach was collected during the field 
test- Data sources Included questionnaires, Interviews, and observations 
throughout the school year. Summary Indices were developed to reduce the 
quantity of data Into simple descriptors of level of Implementation at the 
principal, teacher aW.oVerall school levels, these summary Indices 
describe whether level of implementation was higl? (index = 3), medium 
(index .= 2), or low (index = 1)' with respect to expectations suggested by 
the program modal. Each index summarizes more specific indices which 
describe principal or teacher attitudes and behavior^ High ratings for 
these specific indices indicate that the teacher or principal behavior or 
attitude is at a level which closely approximates an ideal implementation 
of the program with respect to that particular variable. Low indices, on 
the other hand, indicate that implementation is well below ideal levels. 
Medium indices fall between these extremes; they suggest that implementa- 
tion, although not perfect, is generally satisfactory but not of suffi- 
cient magnitude to warrant a high rating. 

Decision rules for the assignment of specific indices were somewhat 
arbitrary. They were essentially based on subjective judgments by the 
program evaluators with respect to what could be reasonably expected of 
principals and teachers under the circumstances of the field test. In 
addition, 'all indices are based on self-report data provided by teachers. 
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principals, and BSC staff. The reliability and validity of these data are 
uncertain. The composition of each summary Index is briefly described 
below. 

Princij)al Level Implementation Index ^.^ 

The' principal level implementation index is comprised of four 
variables, as described in Table D-~l. A specific index was derived for 
each variable. The specific indices, when added together, form the basis 
for the overall principal level index for each school. Sums of the fbur 
indices totaling ten or more were arbitrarily assigned high ratings, sums 
of eight or nine were assigned medium ratings, and sums of seven or below 
were given low ratings. Since results are very subjective, they should be 
viewed cautiously. In a sense, they describe some relative differences 
between principals. 

' . ; ■ ' ' 

Teacher Level Implementation Index 

The teacher level implementation index is comprised of six variables, 

-as described in Table D-2. A specific index was derived for each variable 

All indices were based on average responses to the Teacher Questionnaire. 

As with the overall principal level index, the overall teacher*" level index 

was based on the sum of all s'pecific indices for. each school. Sums of the 

six indices totaling 15 or more were arbitrarily assigned high"- ratings, 

sums between 10 and 14 were assigned medium ratings, and sums of nine or ^ 

below were given low ratings. Again, results are subjective and should be 

cautiously interpreted. The summary index provides a general description 

of the relative differences between teachers at the specific schools. 

2)i ■ 

208 






Table D-1 „ 

Description of Principal Level Implementation Indices 









Decision Rule for Index 


Variables 


Description 


? 

Data 
Sources 


Low 
1 


Medium 
2 


High 
3 


Numbe^r of 
clasiSroom 
observations 


Average number of timfes principal 
observed^ each teacher's classroom; 
sfx were planned 


Teacher questionnaire 


x<6-0- 


6.0<x<7.0 


x>7.0 


Number of 
prmcipax/ Leacnei. 
conferences 


Average number of conferences with 
each teacher 5 three were planned 


Teacher questionnaire, ^ 
principal/ teacher 
conference form 


x<3.0 


3-05x<3.7 


x>3.7 


Training results 


Total index represents the average 
of six specific indices, three for 
the content workshop and three for 
the time workshop. These workshop 
indices include measures of (1) 
overall reaction to training, (2) 
expectations regarding likelihood 
of success, and (3) perceived 
knowledge and skill development 


bleacher training 
questionnaires 


x<3.4 


3. 4<x<4. U 


A 0 
X«^'+ • u 


Attitude toward 
program 


General af feet , enthusiasm 


BSC contacts/observa-- 
tions, interviews, 
principal questionnaire 


^^BSC 
j udgmentl 


BSC 
judgment 


BSC 
judgment 



^ For training results, the decision rules were^-app 
~ overall index. 



li€Ki- t-e' the-si:x"^eclf ic Trmices ^ w^ make up the 



r 
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Table D-2 

Description o^Teacher Level Implementation Indices 











Decision Rule for 


Index. 






Variables 


Description . 


Data 
Sources 


Low 
1 


Medium 
2 


Hig^b 
3 


• - 




Research use 


Combination of measures of teachers', 
reported use of rc?.search/classroom data 
to set improvement goals and improve 
teaching . 


Teacher 
questionnaire 


x<50% 

positive 

responses 


50%<.x<70% 

positive 

responses 


x>70% 

positive 

responses 






* Instructional 
improvements 


Combination of measures of teachers* 
reported improvements in student engaged 
time J content overlap, prior learning, 
student success > mastery, and content review 


Teacher 
questionnaire 


Same 


Same 


Same 




O 


Success in 
. imp lemen t at ion 

J; 


Combination of measures of teachers' 
reported success in implementing Achieve- 
ment Directed Leadership components 
vckiot-oH thi» QfTiidpnt* ensaeed time, content 
overlap, prior learning, student success, 
mastery, and content review 


Teacher 
questionnaire 


Same 


Same 


Same 






Ji : — ■ — 

Changes in 
teaching ^ 
behavior 


Pi^-mKn T-i a t-n o-n o-f TTip ;i cjii TP q of teachsrs ' 
reported changes in instructional planning, 
, classroom management, and teaching 
techniques 


Teacher 
questionnaire 


Same 


Same 


Same 




2 


--impTOWd class- 
room ef f ective-\ 
' ^ess 


Teachers* overall perception of effective- 
ness of basic skills instruction relative 
to prior year 


Teacher 
questionnaire 


Same 

it 


Same 


Same g 


V5 


. Attitude toward 
program 


Teachers' o\^erall reaction; ^niean rating; 
scale of 1 (very negative) to. 5 (very 
positive) \ ' 


Teacher 
questionnaire 


x<3.0 


3.0<x<3.5 


x^3.5" 


'J 


El 







. ' Overall Schoolwide Implementatdon^ "^ .^ridejc 

The overall schoolwide implementation index repres^ts the'sum of the 
four specific principal level indices and the six teacher leVel indices. 
Thus, the overall summary index is influenced more by teacher behaviors 

0 

than by principal behaviors. The rationale for this weighting scTiema is 
that classroom level implementation (i.e., teacher) will, most likely, 
have a greater effect on student achievement. Overall sums of the ten 
indices totaling 25 or more v/ere assigned high ratings, sums between 18 
artd 24 were assigned medium ratings, and sums of- 17 or below were given 
' low overall ratings. Since a large quantity of data went into each 
overall schoolwide index, these indices are probabl^^irly good estimates 
of the relative J.evel af^.T^ for each school within a 



district. 




c 
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Appendix E: 

Self-Assessment Results 

District/Principal Workshops - 
New Jersey 'School^ District 
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Self Assessment Results: Contei .Workshop 







-1 




Dbjectlve (N.= ) ^ ' 


On my 
own 


With 

further 

study 


With some 
assistance 


Only with 
further 
' instruction and 


Even with N 
further / 
instruc t-- c 
ion» may 
not be able 
to do 


ot 

ippli- ! 
lable 












assistance 

- 






Define prior learning, .instructional overlap, and 
mast^ery. (n«2A)* 




17% 


25% 


4% 


U/a 


0% 


.3 . 


Use standardized achievement test data to project 
the classroom average percentile score that can be 
expected under "normal" instruction, (n='2^) 


37% 


37% 


217. 


^% 




0% 


• 


Use test data to identify a class's specific prior 
learning strengths and weaknesses, (n=.25) 




36% 


. 20% 


• 0% 


0% 


0% 




Clnd the overlap between the content taught and the 
content measured by the test. (^=25) 




39% 


12% 


8% ' 


0% 


0% 




Read an instructional overlap graph (find the e;y)ected 
achievemeat zone for a given percent of instructional 
overlap. > (n«26) 


36% 




32% 


O/a 


0% 


0% 




c..^_j_^^'4n-i»--(ni ^f-»^?^->-Mr»^-^^\^»a^ nlfln And Hpr id e whe t her 
CiXamlne iniLiai inscrucL londi. pxcm anu uct^xuc win-i-ii«-»- 

resulting overlap is appropriate for a specific 
|j|uatioa, f„„25) 


36% 




32% . 


8% 






i 


Hpist teachers in completing long-term instructional 
plans, {n='25) 


^0% 


' 2^% 




12% 


0% 


0?: 




Assist teachers in completing unit plans. (n=25) 


^8% 


"20% 


28% . 




/ 0% 






Help teachers review their progress on ttie long-term 
plan and their students* mastery of the topics co^er^^. 


36% 


20% 


^0% 


k% 


0% 


05C 




Assist teachers in revising their long-term plans, if 
necessary, (n=25) 


A0% 


16% ^ 


^0% 


4% 


0% 


0% 
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Self Assessment Results! Planning Seminar 
District/Principals: New Jersey. School District 



Skill (n ^ ) 


On my own 


With 

further 

study 


With 
some 

assistance 


Onlv with 
.further 
instruction 
and 

assistance 


Even with 

further 

instruction 

may not 

be able to do^ 


xCciV.liC'L v'L^ XtS 11 La IxXUIl 








• 




Locate rationale, strategy, content outline (19) 




32% 


21% 






Locate handouts (19) 


m 


37% 


16% 






Locate transparency masters (19) 


37Z 


A7% 


' , 11% 


5% 




Present information from outline (19) 


37Z 


A7% 


16% 






Discuss staff roles and functions (20) 


uo% 


A0% 


20% 






Answer questions (19) 


317. 


32% 


32% 




■ : i 


Follow-up concerns (19) 


267. 


37% 


37% 






Content Implementation \^ 












Use Content Leader's Guide (13) 


15% 


69% 


8% 


rt% 




Time Implementation 












Use Time Leader's Guide (23). 


A 3% 


39% 




9% 
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Self-Assessment Results: Time Woxkshop 
Distact/Principal: New Jersey School District 





)bjective (N - ) 


On my 
own 

• 


With, 
plan- 
ning 


With some 
assistance 


Only with 
further 
instruct- 
ion and 
assistance 


Even with 
further 
instruction, 
may not be 
able to do 


Not 

applic- 
able 


i 


Define allocated time, engagement rate, and 
student engaged time, (n-25) 


60Z 




32% 




0% 


A% 


Distinguish engaged from unengaged student 
behaviors in the classroom, (n=23) 


83% 




13% 


0% 


0% 


UX 


Record data on the engagement rate form and 
calculate engagement rate, (n-23) 






17% 


A% 


Kit* 


A% 


Complete an allocated time log, (n«2A) ^ 


15% 




21% 


0% 


0% " 


A% 


Read and interpret one of the tlnje graphs (find 
the expected achievement zone for a given amount 
of student engaged time,) (n«2A) - 


63X 


* 


33% 


0% 


0% 


A% 


Establish an appropriate goal for student 
engaged time, (n»25) 


lUX 


28% 


uux 

> 


0% 


^ 0% 


A% 


■kablish the corresponding goals for allocated 

an4 engagement rate '(for a given student 
H^ged time goal,) (n°20) 


30% 


A0% 


20% 


5% 


0% 


5% 1 

1 


Assist teachers in selecting appropriate strate- 
gies for increasing allocated time and engagement 
rate, (n-25) 


28% 




20% 


UX 


0% 


A% 


iHelp a teacher plan the implementation of a 
strategy to improve classroom management of students 
and management of instruction, ^ (n-26) 


38% 


3*5% 


19% 


kX 


0% 


A% 


Schedule the initial round of classrood observations. 


'58% 


27% 


8% 


k% 


0% 


A% 


Help teachers assess the impact of their changes on 
classroom practice, especially student engaged time, 

•(n-26) 


38% 


35% 


19% 


UX 


0% 

• 


A% 






Appendix F: 
District Checklists 
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. District Linker Checklist for Time 

insure/secure district support for building participation. 

• Number of buildings + teachers. 

Resources needed (e.g. release tl.e. §. inservice credit) 

. Administrative sanction (superintendent's approval 
recommended) • 

Clarify district parameters of program participation 
(e.g. teachers must volmteer) • 

PlaJ and deliver awareness about time for principals . 



^ 



o o 

} rr H o' 

«^ 5^ « 

p. Oq Dl. 

O 2 



Determine the recipients and mode of the district training-district • 
personnel, principals, or teachers. 

• Formation of an IIT . ' 

. Identify ^ roles and resppnsibilities at each level. 
Develop an" overall plan for SET training. 

• Schedule dates and times. 

• Budget training time. 

.-la. an. conduct gaining sessions as ou.Une. in Lesd^^l^GHi^^MB.- 

• Prepare agendas . 

. secure videotape players . overhead projectors and calculators . 
. Learn to operate a videotape player-including locating different 
parts of the videotape . 

• Reproduce handouts and transparencies. 

■ • Notify participants of meetings, . ^ . . 
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Devise a strategy for assessing participant's mastery of engagement 
rate observation training. : 

- grade own 
hand in/assess 

Share with principals the ramifications of the decisions to who 
will actually perform the classroom observations, the number of 
observations and the length. 

- training time 

- scheduling of observations 

- affect on pbservees 

- match with school climate clinical supervision 

peer supervision 
^ constant observer for one teacher, a variety of observers 

- budgeting or covering 

- number pf cycles 

Discuss the use of codes on data collection forms. 

Provide principals with strategies that can be used to release 
teachers for data collection. 

- rotating substitutes 

- free period observation 

- principal subs in observer's room 

Share with principals the ramifications that school instructional 
groupings and pullout programs have on 

- training decisions 

- observation collection 

Devise a strategy for dealing with resource guides /availability for 
phase three-idea of program and teacher centering links to ISA 
_ and/or local teacher training institutions. 

Deal with the impact that data collection may have upon district/ 
state mandates about allocated time for subj^Ect areas and for pull- 
out programs • ' . 

Consider long-term versus short-term change strategies— need to make 
other changes (perhaps more radical) if SET remains in an unsatxs- 
;factory area. 
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/ -.J ..f Jl.t-rfct needs to evaluate the program and plan for such 
evalun'Lnl data n^ded, how collected, by vhon,, when, and how to 
report • 

5. ■Prepare and assist the principal td perform the fx^ictions listed o 
the principal's checklist. 
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Appendix G: 
Principal Checklist 
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Principal ' s Checklist 
Time, p. 1 



Principal's Checklist for 
Implementatioil of Time Variable^ 

Leading v 

Theo principal ±s^ the instructional leader for the school building. It is 
the principal's responsibility to carry out or to delegate (but maintain the 
responsibility for) each of the tasks/activities listed below. As noted in 
section ll of this guide, the principal's personal participat4.on and encour- 
agemenjt are extremely important. Thus, there will "be some activities that 
the principal should not delegate. 

Planning Collaboratively 

1. Review district ' s overall plan for , 
implementation of the time variable 
(e#g# , who, obsfervefiP, frequency of 
observations, etc.), 

2. _ Identify date for teacher workshops 

^for new or previously untrained 
teachers. Estimate the number of 
teachers involved, 

3. Develop a plan for time workshops for 
teachers (perhaps with a teacher 
committee). 

4. Devise a 'plan for dealing with in- 
service needs of teachers that are 
identified in the Phase Three activi- 
ties. Will principal, district, or 
regional education ag2ncy assist in 
providing for these needs? 

5. Announce date, place, time, and pur- 
pose of workshop. ' 

a. Relate workshop activities to 
overall district and school 
goals. 



si 

^Principals are reminded that in addition to the notes provided at the end 
of this session the Time Leader's Guide will answer many -specific ques- 
tions ab'ut methods and materials. 
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^. ^Principal's Checklist 

Time,^ p. 2 

b. Inform participants of materials 
they need to bring. 

Training , , " 

h* Learn to operate district videotape 
equipmenf—including locating dif- 
ferent parts of the videotape and 
answer keys for coding tapes, 

< <, « 

2. Make last minute check on leaders, 
video equipment , materials , facili- 
ties, notice to teachers, , 

'3, Conduct teacher workshop as outlined 
in planning sessions with district 
staff. Outcome^ of workshop should • 
include . . 

a. an understanding among teachers 
of the goals and expected bene- 
fits of participation in time 
variable activities for the 

school and^ the individual ^ 
teacher. 



b. an understanding of the rationale 
and critical variables (engage- 
ment rate^ allocated time, stu- 
dent engaged time', and student 
success) related to the time 
variable, 

c. an understanding- of what is meant 
by engaged and unengaged behav- 
iors and what observers Will be 
coding and calculating on the 
engagement rate form, 

d. knowledge of how and when to com- 
plete an allocated time log, 

e. an understanding of the distinc- 
tion betwee'n managing students 
and managing instruction; knowl- < 
edge of location of strategies 
for improving engagement rate and 
allocated time, 
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Principal's Checklist 
Time, p. 3 




f, observation schedule for the ini- 
tial round of classroom observa- 
tions, and 

g. get an understanding of the 
expectations the principal Jias 
and tlie procedures the principal 
will follow in working with 
teachers on the data .^that will 
be collected, 

4. Evaluate teacher workshop (see evalu- 
ation form that follows this list). 




Participating • - 

1. Develop and implement a way of re- 
minding teachers to complete allo- 
cated time ,logs on observation days* 

2. Carry out observation schedule (ini- 
tial and, follow-up) collecting" 
engagement rate data • 

3. -Help teachers complete summary sheets 
and set goals for student engaged 
time, allocated time, and engagement ■ 
rate. 



4. Help teachers select, implement, and 
monitor strategies . to meet goals, 

5. Implement any identified school-wide 
strategies concerning allocated time 
or engagement rate, 

6. Work with individual teachers whose 
student engaged- time remains unsatis- 
factory. Consider long-term versus 
short-term change strategies — need to 
make other chang^ (perhaps more 
radical) if student engaged time 
remains in an unsatisfactory area; 

7. Synthesize teacher ^inservicq, and pro- 
fessional growth needs based on data 
collected from plans and consulta- 
tions with teachers, 

r 
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8. ' Arrange inservice by district or re- 

gional agency for groups of teachers 
with same ^Interest or need. 

9, Attendi dis tricf^^meeting to learn 
additional strategies or techniques 
for working with teachers. 

10. Communicate to district any observed 
scheduling (pullouts^. special 
classes) procedures that lower allo- 
cated time for buildings/classrooms. 

11. Collect information needed to review 
success of time varirable implemen- 
tation. 



Principal's Checklist 
Time, p. 4 " ^ 



a. 



b. 

r 



Collect teachers' completed 
Summary Sheets at end of year. 

Analyze process and achievement 
data for each instructional 
group. 

- student engaged time 

- percentile ranks 
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Principal's Checklist 
Content, p. 1 

" ■ ^ 

Principalis Checklist for Implementing 
Instructional Planning (Content Variables) * 

Leading . * 

The principal is^ the* instructional leader for the school building. It is 
the rii^incipal's responsibility to carry out or to delegate (but maintain 
respSisibillty for) each of the tasks/activities listed below. As noted in 
the previous section of this guide , the ^principal ' s personal participation 
and ehcouragement are extremely important. Thus there will be some activi- 
ties !the principal should not delegate. 

Planning Collaboratively 

1. Review district's overall plan* for 
systematic' instructional planning. 

2. Identify August or September date(s; 
for teacher,; workshop(s). Estimate 
number of teachers involved; 

3. Identify (with district) source of 
prior learning data (test data or 
pupil profiles). 

4. Identify (with district) materials _ 
to be used in work on instructional 
overlap (test descriptions, district 
curriculum guide, basal text). 

5. Assist district with any needed 
curriculum-test matching activities. 

6. Develop a plan for content workshop 
for teachers (perhaps with teacher 
committee) . 

a. Determine scope and objectives of _ 
workshop; distinguish between 
workshop activities and outside 
assignments. 

*Principals are -reminded that in addition to the notes provided at the end 
of this session, the Content Leader's Guide will answer many specific ques 
tions about methods and materials. ♦ 



/ 
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Principalis. Checklist 
Content, p. 2 



b. Decfide who will lead each part of 
workshop — notify individuals. 

* 

c. Arrange for necessary materials • 
(agendas, data,, transparencies, 
handouts) . 

d. Determine room and equipment to 
be used (a lot of table space is 
needed , overhead pr objector , 
ioffee?). 

?• Devise a strategy for dealing with 
identified training needs--idea of 
. program and teacher centering links 
to district, regional education 
" agency, and/or local teacher training 
institutions • . 

8. Announce date, place, time, and pur- 
pose of workshop, 

a. Relate workshop activities to 
overall district and school 
goals. *^ 

b. Inform participants of materials 
they' need to bring. 



Training 

V. Complete a last minute check on 
leaders, materials, facilities, 
notice to teachers. 



2. Conduct teacher workshop as outlined 
in planning sessions with district 
staff. 

3. Evaluatle teacher workshop (see evalu- 
ation form that follows this list). 



Participating 

!• Identify and implement strategies to 
address major prior learning 
deficiencies. 
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Principal's Checklist 
Content , p. 3 



2. Meet with small groups or ^ individual 
teachers to monitor progress on 
School Year Planning Guide, and 
possibly review Unit Topic Plans and 
status of class mastery — weekly or 
every other week as lesson plans are 
checked^ ^ — - _ — 

.3. Tell district about curriculum v/eak- 
nesses determined in pricrt' learning 
analysis. (This may serve as input 
for curriculum revision processes.) 

4. Synthesize teacher inservice and, pro- 
V fessional growth needs based on data 

collected from plans and consulta- ^ 
tions with teachers. 

5. " Arrange inservice by district or 

regional agency for groups of 
teachers with same intere&t or need. , 

6. Attend district meeting to Team 
additional strategies and techniques 
fof helping teachers achieve coverage 
goals. 

7. Collect information needed to review 
success of "instructional planning 
effort. 

a. Collect teachers* completed 
Content Summary' Sheets prior to 
administering^ testing program. . 

b. Analyze process and achievement 
data for each instructional 
group. . w - . _ 

- intended and actual instruc- 
tional overlap - 

- percentile ranks 
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